[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:08 dist-test-slave-6k22 krb5kdc[22545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:08 dist-test-slave-6k22 krb5kdc[22545](info): set up 2 sockets
Aug 08 07:11:08 dist-test-slave-6k22 krb5kdc[22545](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 07:11:09.876956 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:09 dist-test-slave-6k22 krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:10.006456 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33135
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:33135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:10.089985 22559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:10.090209 22559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:10.090233 22559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:10.091439 22559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:10.091490 22559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:10.091504 22559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:10.091517 22559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:10.091531 22559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:10.093283 22559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:33135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:10.093600 22559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:10.093856 22559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:10.099337 22565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:10.099357 22564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:10.099411 22567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:10.099396 22559 server_base.cc:1034] running on GCE node
I20260808 07:11:10.099817 22559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:10.100080 22559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:10.101217 22559 hybrid_clock.cc:648] HybridClock initialized: now 1786173070101225 us; error 68 us; skew 500 ppm
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:10.103804 22559 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:10.104270 22559 webserver.cc:460] Webserver started at http://127.22.2.190:43255/ using document root <none> and password file <none>
I20260808 07:11:10.104504 22559 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:10.104557 22559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:10.104723 22559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:10.105882 22559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "3c3726fa36c745d8a54fc91d1a2837cd"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "e2fdcd89084ed836ae784f268848e59f"
server_key_iv: "bf3267ff65df89615e6c003c5b81479f"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.106246 22559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "3c3726fa36c745d8a54fc91d1a2837cd"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "e2fdcd89084ed836ae784f268848e59f"
server_key_iv: "bf3267ff65df89615e6c003c5b81479f"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.107606 22559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:10.108444 22574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.108620 22559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:10.108711 22559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "3c3726fa36c745d8a54fc91d1a2837cd"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "e2fdcd89084ed836ae784f268848e59f"
server_key_iv: "bf3267ff65df89615e6c003c5b81479f"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.108781 22559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:10.119721 22559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:10.121877 22559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:10.122040 22559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:10.126610 22559 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:33135
I20260808 07:11:10.126683 22629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:33135 every 8 connection(s)
I20260808 07:11:10.127027 22559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:10.128043 22630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:10.130686 22630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd: Bootstrap starting.
I20260808 07:11:10.131349 22630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:10.131723 22630 log.cc:824] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:10.132115 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 22559
I20260808 07:11:10.132234 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:10.132546 22630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd: No bootstrap required, opened a new log
I20260808 07:11:10.132892 22538 external_mini_cluster.cc:1428] Setting key c8d7e7a32264f21c8452650ca262cfb5
I20260808 07:11:10.134883 22630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3726fa36c745d8a54fc91d1a2837cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33135 } }
I20260808 07:11:10.135049 22630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:10.135130 22630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3726fa36c745d8a54fc91d1a2837cd, State: Initialized, Role: FOLLOWER
I20260808 07:11:10.135305 22630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3726fa36c745d8a54fc91d1a2837cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33135 } }
I20260808 07:11:10.135404 22630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:10.135452 22630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:10.135504 22630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:10.136456 22630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3726fa36c745d8a54fc91d1a2837cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33135 } }
I20260808 07:11:10.136582 22630 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c3726fa36c745d8a54fc91d1a2837cd; no voters:
I20260808 07:11:10.136777 22630 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:10.136802 22635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:10.136927 22635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [term 1 LEADER]: Becoming Leader. State: Replica: 3c3726fa36c745d8a54fc91d1a2837cd, State: Running, Role: LEADER
I20260808 07:11:10.137015 22635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3726fa36c745d8a54fc91d1a2837cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33135 } }
I20260808 07:11:10.137053 22630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:10.137359 22635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c3726fa36c745d8a54fc91d1a2837cd. Latest consensus state: current_term: 1 leader_uuid: "3c3726fa36c745d8a54fc91d1a2837cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3726fa36c745d8a54fc91d1a2837cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33135 } } }
I20260808 07:11:10.137439 22635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:10.137382 22637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c3726fa36c745d8a54fc91d1a2837cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3726fa36c745d8a54fc91d1a2837cd" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33135 } } }
I20260808 07:11:10.137614 22637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:10.137682 22641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:10.138298 22641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:10.140163 22641 catalog_manager.cc:1347] Generated new cluster ID: 8e0008a59c7e4b4a963ab29ac8defbef
I20260808 07:11:10.140219 22641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:10.149829 22634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.133841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46430 (local address 127.22.2.190:33135)
0808 07:11:10.134179 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:10.134188 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:10.134363 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:11:10.136386 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:10.136391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:10.136431 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:10.136552 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:10.139007 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.140127 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.141474 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.142042 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.142108 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.145725 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:10.145733 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:10.145734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:10.145744 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:10.148028 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.148454 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.148457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.148457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.148561 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.148817 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.148818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.148819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.149046 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:10.149202 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:10.149467 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 07:11:10.149607 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":217,"threads_started":1}
I20260808 07:11:10.152460 22641 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:10.152989 22641 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:10.157310 22641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c3726fa36c745d8a54fc91d1a2837cd: Generated new TSK 0
I20260808 07:11:10.157474 22641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:10.195434 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33135
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:10.289568 22657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:10.289834 22657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:10.289886 22657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:10.291049 22657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:10.291115 22657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:10.291163 22657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:10.292778 22657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33135
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:10.293035 22657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:10.293277 22657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:10.299856 22663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:10.299957 22665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:10.299870 22662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:10.300122 22657 server_base.cc:1034] running on GCE node
I20260808 07:11:10.300321 22657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:10.300599 22657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:10.301759 22657 hybrid_clock.cc:648] HybridClock initialized: now 1786173070301738 us; error 37 us; skew 500 ppm
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:10.304626 22657 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:10.305051 22657 webserver.cc:460] Webserver started at http://127.22.2.129:43435/ using document root <none> and password file <none>
I20260808 07:11:10.305300 22657 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:10.305365 22657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:10.305508 22657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:10.306567 22657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "3232e30928a94622a0d496102fe930df"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "67739a794c87fb19f9a6b765d896792e"
server_key_iv: "91370323b0ce251a46e87ac8801406e2"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.306932 22657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "3232e30928a94622a0d496102fe930df"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "67739a794c87fb19f9a6b765d896792e"
server_key_iv: "91370323b0ce251a46e87ac8801406e2"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.308436 22657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:10.309226 22673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.309455 22657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:11:10.309556 22657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "3232e30928a94622a0d496102fe930df"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "67739a794c87fb19f9a6b765d896792e"
server_key_iv: "91370323b0ce251a46e87ac8801406e2"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.309625 22657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:10.325397 22657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:10.327445 22657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:10.327626 22657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:10.327859 22657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:10.328166 22657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:10.328217 22657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.328266 22657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:10.328295 22657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.334676 22657 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:37159
I20260808 07:11:10.334808 22794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:37159 every 8 connection(s)
I20260808 07:11:10.335117 22657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:10.342020 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 22657
I20260808 07:11:10.342120 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:10.342312 22538 external_mini_cluster.cc:1428] Setting key 4d59b05366add133d38c9d4ff2bc5304
I20260808 07:11:10.345474 22634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.335934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:36927 (local address 127.22.2.190:33135)
0808 07:11:10.336015 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:10.336018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:10.336242 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:11:10.336835 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:10.336836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:10.336854 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:10.336912 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:10.338254 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.338779 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.339951 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.340145 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.340194 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.342319 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:10.342321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:10.342321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:10.342328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:10.344070 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.344464 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.344465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.344466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.344533 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.344834 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.344835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.344835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.344954 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:10.345025 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:10.345293 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 07:11:10.345380 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:11:10.345669 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.335893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33135 (local address 127.22.2.129:36927)
0808 07:11:10.336089 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:10.336147 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:10.336718 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:10.337101 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:10.337105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:10.337810 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:10.338055 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:10.338063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.339044 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.339045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:10.339871 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:10.339874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.340221 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.340222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:10.340325 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.340780 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:10.340789 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:10.342233 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:10.344148 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:10.344154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:10.344156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:10.344378 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:10.344600 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:10.344601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:10.344601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:10.344753 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:10.345026 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:10.345035 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:10.345167 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 07:11:10.345270 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:11:10.345425 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260808 07:11:10.346215 22795 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33135
I20260808 07:11:10.346318 22795 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:10.346514 22795 heartbeater.cc:511] Master 127.22.2.190:33135 requested a full tablet report, sending...
I20260808 07:11:10.347041 22592 ts_manager.cc:194] Registered new tserver with Master: 3232e30928a94622a0d496102fe930df (127.22.2.129:37159)
I20260808 07:11:10.347728 22592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:36927
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:10.370249 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33135
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:10.454864 22801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:10.455117 22801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:10.455174 22801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:10.456282 22801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:10.456318 22801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:10.456346 22801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:10.457675 22801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33135
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:10.457857 22801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:10.458021 22801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:10.463106 22806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:10.463238 22807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:10.463267 22801 server_base.cc:1034] running on GCE node
W20260808 07:11:10.463105 22809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:10.463653 22801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:10.463925 22801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:10.465081 22801 hybrid_clock.cc:648] HybridClock initialized: now 1786173070465066 us; error 32 us; skew 500 ppm
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:10.467810 22801 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:10.468379 22801 webserver.cc:460] Webserver started at http://127.22.2.130:35705/ using document root <none> and password file <none>
I20260808 07:11:10.468662 22801 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:10.468717 22801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:10.468874 22801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:10.469940 22801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "193dac9d376b4ca6a753f4114f8f0be4"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "68d59ca5927fb25b617dbb8ad8e3f0b7"
server_key_iv: "38dbc830be40709116f39f3da18fc132"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.470323 22801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "193dac9d376b4ca6a753f4114f8f0be4"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "68d59ca5927fb25b617dbb8ad8e3f0b7"
server_key_iv: "38dbc830be40709116f39f3da18fc132"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.471896 22801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:10.472751 22816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.472957 22801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:10.473049 22801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "193dac9d376b4ca6a753f4114f8f0be4"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "68d59ca5927fb25b617dbb8ad8e3f0b7"
server_key_iv: "38dbc830be40709116f39f3da18fc132"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.473109 22801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:10.487322 22801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:10.489833 22801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:10.489961 22801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:10.490172 22801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:10.490446 22801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:10.490481 22801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.490507 22801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:10.490521 22801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.497017 22801 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:45337
I20260808 07:11:10.497098 22936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:45337 every 8 connection(s)
I20260808 07:11:10.497426 22801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:10.505867 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 22801
I20260808 07:11:10.505992 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:10.506179 22538 external_mini_cluster.cc:1428] Setting key 42ffb68fb85598714b5791a0f2c9da9d
I20260808 07:11:10.507148 22634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.498200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:56499 (local address 127.22.2.190:33135)
0808 07:11:10.498264 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:10.498267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:10.498432 (+ 165us) server_negotiation.cc:407] Connection header received
0808 07:11:10.498918 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:10.498919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:10.498930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:10.498997 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:10.500016 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.500580 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.501624 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.501893 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.501958 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.504082 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:10.504083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:10.504083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:10.504090 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:10.505527 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.505939 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.505940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.505941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.506015 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.506353 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.506354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.506354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.506510 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:10.506585 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:10.506949 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 07:11:10.507039 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:11:10.507246 22939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.498117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33135 (local address 127.22.2.130:56499)
0808 07:11:10.498294 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:10.498337 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:10.498831 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:10.499070 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:10.499073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:10.499701 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:10.499926 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:10.499932 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.500754 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.500755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:10.501535 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:10.501537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.501974 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.501975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:10.502079 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.502531 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:10.502539 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:10.503993 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:10.505609 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:10.505616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:10.505617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:10.505856 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:10.506107 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:10.506108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:10.506108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:10.506229 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:10.506590 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:10.506598 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:10.506794 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 07:11:10.506908 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:11:10.507060 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":107,"threads_started":1}
I20260808 07:11:10.507778 22937 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33135
I20260808 07:11:10.507885 22937 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:10.508057 22937 heartbeater.cc:511] Master 127.22.2.190:33135 requested a full tablet report, sending...
I20260808 07:11:10.508462 22592 ts_manager.cc:194] Registered new tserver with Master: 193dac9d376b4ca6a753f4114f8f0be4 (127.22.2.130:45337)
I20260808 07:11:10.508987 22592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:56499
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:10.534420 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33135
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:10.617810 22942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:10.618005 22942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:10.618028 22942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:10.619175 22942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:10.619226 22942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:10.619253 22942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:10.620729 22942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33135
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:10.620972 22942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:10.621199 22942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:10.626291 22950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:10.626276 22954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:10.626314 22947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:10.626503 22942 server_base.cc:1034] running on GCE node
I20260808 07:11:10.626710 22942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:10.626915 22942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:10.628057 22942 hybrid_clock.cc:648] HybridClock initialized: now 1786173070628036 us; error 40 us; skew 500 ppm
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:10.630746 22942 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:10.631189 22942 webserver.cc:460] Webserver started at http://127.22.2.131:44901/ using document root <none> and password file <none>
I20260808 07:11:10.631390 22942 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:10.631429 22942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:10.631522 22942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:10.632680 22942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "54baa156ff8e4a2fb90571290ec21573"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "f6ef1d37f746614037601f871cbcff18"
server_key_iv: "e954e66d845cb69bb8729d7d60c249e8"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.632978 22942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "54baa156ff8e4a2fb90571290ec21573"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "f6ef1d37f746614037601f871cbcff18"
server_key_iv: "e954e66d845cb69bb8729d7d60c249e8"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.634395 22942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:10.635195 22961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.635402 22942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:10.635499 22942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "54baa156ff8e4a2fb90571290ec21573"
format_stamp: "Formatted at 2026-08-08 07:11:10 on dist-test-slave-6k22"
server_key: "f6ef1d37f746614037601f871cbcff18"
server_key_iv: "e954e66d845cb69bb8729d7d60c249e8"
server_key_version: "encryptionkey@0"
I20260808 07:11:10.635601 22942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:10.655505 22942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:10.657757 22942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:10.657894 22942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:10.658116 22942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:10.658828 22942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:10.658867 22942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.658917 22942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:10.658946 22942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:10.665964 22942 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:37041
I20260808 07:11:10.666045 23090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:37041 every 8 connection(s)
I20260808 07:11:10.666404 22942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:10.669909 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 22942
I20260808 07:11:10.670023 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:10.670173 22538 external_mini_cluster.cc:1428] Setting key dcc5371ddd6c4b6a1d4a35ad3696d532
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:10.677187 22634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.667258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:48915 (local address 127.22.2.190:33135)
0808 07:11:10.667308 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:10.667310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:10.667522 (+ 212us) server_negotiation.cc:407] Connection header received
0808 07:11:10.668219 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:10.668221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:10.668235 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:10.668279 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:10.669372 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.669965 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.671522 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.671800 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.671891 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.674349 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:10.674351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:10.674352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:10.674360 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:10.675741 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.676164 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.676165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.676165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.676252 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.676508 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.676509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.676510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.676630 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:10.676704 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:10.677006 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 07:11:10.677086 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:10.677372 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.667225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33135 (local address 127.22.2.131:48915)
0808 07:11:10.667388 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:10.667429 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:10.668099 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:10.668360 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:10.668364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:10.669035 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:10.669262 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:10.669268 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.670091 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.670092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:10.671413 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:10.671416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.671905 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.671906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:10.672084 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.672684 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:10.672692 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:10.674223 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:10.675836 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:10.675843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:10.675845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:10.676075 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:10.676333 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:10.676334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:10.676334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:10.676442 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:10.676697 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:10.676705 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:10.676841 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 07:11:10.676947 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:11:10.677130 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260808 07:11:10.677824 23091 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33135
I20260808 07:11:10.677930 23091 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:10.678123 23091 heartbeater.cc:511] Master 127.22.2.190:33135 requested a full tablet report, sending...
I20260808 07:11:10.678495 22592 ts_manager.cc:194] Registered new tserver with Master: 54baa156ff8e4a2fb90571290ec21573 (127.22.2.131:37041)
I20260808 07:11:10.678879 22592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:48915
I20260808 07:11:10.682106 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:10.706483 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.698698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.22.2.129:37159)
0808 07:11:10.698894 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:10.698898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:10.698910 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:11:10.698937 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:10.698939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:10.698953 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:10.699023 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:10.699630 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.700371 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.701052 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.701373 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.701427 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.703749 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:10.703759 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:10.703760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:10.703767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:10.705402 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.705752 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.705755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.705755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.705809 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.705967 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.705968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.705968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.706099 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:10.706222 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:10.706283 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 07:11:10.706346 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
W20260808 07:11:10.706857 22702 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48474
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:10.730011 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:10.723087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.22.2.129:37159)
0808 07:11:10.723181 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:10.723183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:10.723212 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:11:10.723268 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:10.723269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:10.723279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:10.723344 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:10.723970 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.724477 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.725185 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:10.725397 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:10.725465 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:10.727710 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:10.727711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:10.727711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:10.727718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:10.729033 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.729372 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.729373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.729374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.729429 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:10.729589 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:10.729589 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:10.729589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:10.729690 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:10.729786 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:10.729853 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:11:10.729916 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 07:11:10.730587 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 22657
I20260808 07:11:10.739952 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 22801
I20260808 07:11:10.751374 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 22942
I20260808 07:11:10.756495 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 22559
2026-08-08T07:11:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2023 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[23108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[23108](info): set up 2 sockets
Aug 08 07:11:10 dist-test-slave-6k22 krb5kdc[23108](info): commencing operation
krb5kdc: starting...
W20260808 07:11:11.903280 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:11 dist-test-slave-6k22 krb5kdc[23108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:12.034651 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34045
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:34045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:12.117995 23123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:12.118270 23123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:12.118314 23123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:12.119500 23123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:12.119598 23123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:12.119638 23123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:12.119668 23123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:12.119704 23123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:12.121196 23123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:34045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:12.121480 23123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:12.121726 23123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:12.126778 23129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:12.126778 23130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:12.126895 23123 server_base.cc:1034] running on GCE node
W20260808 07:11:12.126801 23132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:12.127310 23123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:12.127664 23123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:12.128818 23123 hybrid_clock.cc:648] HybridClock initialized: now 1786173072128772 us; error 66 us; skew 500 ppm
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:12.137398 23123 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:12.137894 23123 webserver.cc:460] Webserver started at http://127.22.2.190:43663/ using document root <none> and password file <none>
I20260808 07:11:12.138126 23123 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:12.138181 23123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:12.138296 23123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:12.139657 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "d0b160552aad4da6a8a020f28d37a404"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "8c56613b9f998588e83e79de3102b51c"
server_key_iv: "667944c6a614d383729e1b6cd93f30e2"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.140107 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "d0b160552aad4da6a8a020f28d37a404"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "8c56613b9f998588e83e79de3102b51c"
server_key_iv: "667944c6a614d383729e1b6cd93f30e2"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.141695 23123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:12.142508 23143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.142763 23123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:12.142896 23123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "d0b160552aad4da6a8a020f28d37a404"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "8c56613b9f998588e83e79de3102b51c"
server_key_iv: "667944c6a614d383729e1b6cd93f30e2"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.142987 23123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:12.163203 23123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:12.165750 23123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:12.165969 23123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:12.171028 23123 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:34045
I20260808 07:11:12.171069 23206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:34045 every 8 connection(s)
I20260808 07:11:12.171494 23123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:12.172446 23208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.176589 23208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404: Bootstrap starting.
I20260808 07:11:12.177452 23208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.177842 23208 log.cc:824] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:12.178828 23208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404: No bootstrap required, opened a new log
I20260808 07:11:12.180439 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 23123
I20260808 07:11:12.180543 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:12.180774 22538 external_mini_cluster.cc:1428] Setting key a67c4b11b5b3afa2c21453f41b289f36
I20260808 07:11:12.180831 23208 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b160552aad4da6a8a020f28d37a404" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34045 } }
I20260808 07:11:12.181039 23208 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.181082 23208 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0b160552aad4da6a8a020f28d37a404, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.181257 23208 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b160552aad4da6a8a020f28d37a404" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34045 } }
I20260808 07:11:12.181356 23208 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:12.181418 23208 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:12.181479 23208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.182528 23208 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b160552aad4da6a8a020f28d37a404" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34045 } }
I20260808 07:11:12.182664 23208 leader_election.cc:302] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0b160552aad4da6a8a020f28d37a404; no voters:
I20260808 07:11:12.182813 23208 leader_election.cc:288] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:12.182885 23213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:12.183039 23208 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:12.183043 23213 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [term 1 LEADER]: Becoming Leader. State: Replica: d0b160552aad4da6a8a020f28d37a404, State: Running, Role: LEADER
I20260808 07:11:12.183173 23213 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b160552aad4da6a8a020f28d37a404" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34045 } }
I20260808 07:11:12.183523 23213 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0b160552aad4da6a8a020f28d37a404. Latest consensus state: current_term: 1 leader_uuid: "d0b160552aad4da6a8a020f28d37a404" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b160552aad4da6a8a020f28d37a404" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34045 } } }
I20260808 07:11:12.183642 23213 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:12.183537 23215 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0b160552aad4da6a8a020f28d37a404" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b160552aad4da6a8a020f28d37a404" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34045 } } }
I20260808 07:11:12.183800 23215 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404 [sys.catalog]: This master's current role is: LEADER
W20260808 07:11:12.184866 23229 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:12.184924 23229 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:12.184983 23222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:12.185124 23222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:12.187084 23222 catalog_manager.cc:1347] Generated new cluster ID: 22b236613ade4904b670d0dede02139c
I20260808 07:11:12.187145 23222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:12.192646 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.181336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47718 (local address 127.22.2.190:34045)
0808 07:11:12.181545 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.181555 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.181588 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:11:12.181725 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.181732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.181829 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.182004 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.182628 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.184130 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.185137 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.185656 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.185810 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.188698 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.188702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.188702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.188715 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.190929 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.191348 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.191352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.191353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.191507 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.191738 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.191739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.191739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.192070 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.192230 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.192301 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.192443 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
I20260808 07:11:12.213150 23222 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:12.213712 23222 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:12.218386 23222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0b160552aad4da6a8a020f28d37a404: Generated new TSK 0
I20260808 07:11:12.218561 23222 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:12.253816 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34045
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:12.338096 23237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:12.338357 23237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:12.338385 23237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:12.339658 23237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:12.339725 23237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:12.339753 23237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:12.341275 23237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34045
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:12.341482 23237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:12.341660 23237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:12.346849 23242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:12.346860 23245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:12.346829 23243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:12.347258 23237 server_base.cc:1034] running on GCE node
I20260808 07:11:12.347473 23237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:12.347735 23237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:12.348899 23237 hybrid_clock.cc:648] HybridClock initialized: now 1786173072348893 us; error 37 us; skew 500 ppm
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:12.351472 23237 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:12.351923 23237 webserver.cc:460] Webserver started at http://127.22.2.129:36905/ using document root <none> and password file <none>
I20260808 07:11:12.352128 23237 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:12.352169 23237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:12.352267 23237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:12.353286 23237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "af08ffd8991542a7af4b97815d3c8e07"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "13bb818a3b79d10f42db394924c9276a"
server_key_iv: "059640297f046482a88522bcef50a56c"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.353596 23237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "af08ffd8991542a7af4b97815d3c8e07"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "13bb818a3b79d10f42db394924c9276a"
server_key_iv: "059640297f046482a88522bcef50a56c"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.354883 23237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:12.355660 23253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.355834 23237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:12.355922 23237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "af08ffd8991542a7af4b97815d3c8e07"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "13bb818a3b79d10f42db394924c9276a"
server_key_iv: "059640297f046482a88522bcef50a56c"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.355978 23237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:12.372601 23237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:12.374579 23237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:12.374697 23237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:12.374897 23237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:12.375258 23237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:12.375293 23237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.375344 23237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:12.375371 23237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.381476 23237 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33395
I20260808 07:11:12.381549 23375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33395 every 8 connection(s)
I20260808 07:11:12.381918 23237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:12.389333 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 23237
I20260808 07:11:12.389448 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:12.389639 22538 external_mini_cluster.cc:1428] Setting key 3991aba01153fb2568f113630ee30d40
I20260808 07:11:12.391980 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.382783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:51597 (local address 127.22.2.190:34045)
0808 07:11:12.382854 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.382856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.383059 (+ 203us) server_negotiation.cc:407] Connection header received
0808 07:11:12.384125 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.384126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.384139 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.384236 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.384829 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.385380 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.386372 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.386561 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.386612 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.388773 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.388775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.388775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.388783 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.390415 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.390870 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.390871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.390872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.390937 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.391199 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.391200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.391201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.391331 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.391402 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.391772 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.391866 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:11:12.392206 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.382733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34045 (local address 127.22.2.129:51597)
0808 07:11:12.382914 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.382960 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.384013 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.384286 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.384289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.384544 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:12.384749 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.384753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.385467 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.385467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.386296 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.386298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.386637 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.386638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.386749 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.387218 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:12.387227 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:12.388682 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:12.390495 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:12.390504 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:12.390506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:12.390778 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:12.391007 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:12.391008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:12.391008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:12.391131 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:12.391405 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:12.391414 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:12.391600 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.391746 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.391935 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260808 07:11:12.392949 23378 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34045
I20260808 07:11:12.393067 23378 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:12.393275 23378 heartbeater.cc:511] Master 127.22.2.190:34045 requested a full tablet report, sending...
I20260808 07:11:12.393805 23162 ts_manager.cc:194] Registered new tserver with Master: af08ffd8991542a7af4b97815d3c8e07 (127.22.2.129:33395)
I20260808 07:11:12.394419 23162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:51597
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:12.418056 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34045
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:12.501588 23384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:12.501842 23384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:12.501885 23384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:12.503038 23384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:12.503108 23384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:12.503153 23384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:12.504698 23384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34045
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:12.504976 23384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:12.505213 23384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:12.510316 23390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:12.510416 23384 server_base.cc:1034] running on GCE node
W20260808 07:11:12.510351 23392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:12.510316 23389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:12.510762 23384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:12.511008 23384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:12.512168 23384 hybrid_clock.cc:648] HybridClock initialized: now 1786173072512158 us; error 47 us; skew 500 ppm
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:12.514717 23384 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:12.515151 23384 webserver.cc:460] Webserver started at http://127.22.2.130:39521/ using document root <none> and password file <none>
I20260808 07:11:12.515399 23384 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:12.515466 23384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:12.515641 23384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:12.516777 23384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "6f2630df56a74bd3b86904fb3ef8ffb0"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "7d7b9e5f0a44d899fac13fb47b1ce473"
server_key_iv: "13fab769b72e04fd7dcb87145ed6e741"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.517138 23384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "6f2630df56a74bd3b86904fb3ef8ffb0"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "7d7b9e5f0a44d899fac13fb47b1ce473"
server_key_iv: "13fab769b72e04fd7dcb87145ed6e741"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.518563 23384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:12.519299 23399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.519501 23384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:12.519616 23384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "6f2630df56a74bd3b86904fb3ef8ffb0"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "7d7b9e5f0a44d899fac13fb47b1ce473"
server_key_iv: "13fab769b72e04fd7dcb87145ed6e741"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.519697 23384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:12.526695 23384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:12.528558 23384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:12.528707 23384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:12.528916 23384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:12.529211 23384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:12.529263 23384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.529304 23384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:12.529331 23384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.535264 23384 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35599
I20260808 07:11:12.535310 23517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35599 every 8 connection(s)
I20260808 07:11:12.535694 23384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:12.543413 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 23384
I20260808 07:11:12.543521 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:12.543720 22538 external_mini_cluster.cc:1428] Setting key 5751b475206ef2b3d0eb159e5136ce59
I20260808 07:11:12.545341 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.536418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:54065 (local address 127.22.2.190:34045)
0808 07:11:12.536467 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.536469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.536679 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:11:12.537602 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.537603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.537614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.537667 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.538234 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.538735 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.539765 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.539959 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.540018 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.542118 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.542119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.542119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.542126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.543711 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.544255 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.544256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.544256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.544324 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.544683 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.544684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.544684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.544822 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.544877 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.545207 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.545278 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:12.545670 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.536381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34045 (local address 127.22.2.130:54065)
0808 07:11:12.536575 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.536610 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.537503 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.537720 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.537723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.537971 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:12.538155 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.538159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.538817 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.538818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.539691 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.539694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.540034 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.540035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.540135 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.540592 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:12.540599 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:12.542032 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:12.543821 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:12.543829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:12.543832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:12.544184 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:12.544404 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:12.544404 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:12.544405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:12.544609 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:12.544892 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:12.544899 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:12.545068 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.545188 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.545386 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":112,"threads_started":1}
I20260808 07:11:12.546223 23518 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34045
I20260808 07:11:12.546339 23518 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:12.546547 23518 heartbeater.cc:511] Master 127.22.2.190:34045 requested a full tablet report, sending...
I20260808 07:11:12.546988 23162 ts_manager.cc:194] Registered new tserver with Master: 6f2630df56a74bd3b86904fb3ef8ffb0 (127.22.2.130:35599)
I20260808 07:11:12.547423 23162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:54065
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:12.571329 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34045
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:12.654156 23524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:12.654362 23524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:12.654388 23524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:12.655450 23524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:12.655485 23524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:12.655511 23524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:12.657080 23524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34045
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:12.657284 23524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:12.657482 23524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:12.662689 23530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:12.662688 23529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:12.662684 23532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:12.662968 23524 server_base.cc:1034] running on GCE node
I20260808 07:11:12.663257 23524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:12.663514 23524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:12.664757 23524 hybrid_clock.cc:648] HybridClock initialized: now 1786173072664731 us; error 50 us; skew 500 ppm
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:12.667423 23524 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:12.667917 23524 webserver.cc:460] Webserver started at http://127.22.2.131:35619/ using document root <none> and password file <none>
I20260808 07:11:12.668162 23524 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:12.668216 23524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:12.668315 23524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:12.669318 23524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "2e7a58316aed4f51aee779a9738c6f29"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "6a442d6df170fd1fdd3619c84e0e0788"
server_key_iv: "82b6469d9348988a4485249772e0d145"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.669610 23524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "2e7a58316aed4f51aee779a9738c6f29"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "6a442d6df170fd1fdd3619c84e0e0788"
server_key_iv: "82b6469d9348988a4485249772e0d145"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.670876 23524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:12.671590 23541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.671774 23524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:11:12.671864 23524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "2e7a58316aed4f51aee779a9738c6f29"
format_stamp: "Formatted at 2026-08-08 07:11:12 on dist-test-slave-6k22"
server_key: "6a442d6df170fd1fdd3619c84e0e0788"
server_key_iv: "82b6469d9348988a4485249772e0d145"
server_key_version: "encryptionkey@0"
I20260808 07:11:12.671921 23524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:12.689409 23524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:12.691521 23524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:12.691694 23524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:12.691895 23524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:12.692216 23524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:12.692257 23524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.692299 23524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:12.692327 23524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.698087 23524 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:39063
I20260808 07:11:12.698233 23665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:39063 every 8 connection(s)
I20260808 07:11:12.698513 23524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:12.706785 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 23524
I20260808 07:11:12.706900 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:12.707090 22538 external_mini_cluster.cc:1428] Setting key 406e0747db5ad735f71c33e264242da2
I20260808 07:11:12.708086 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.699283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:54555 (local address 127.22.2.190:34045)
0808 07:11:12.699329 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.699332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.699575 (+ 243us) server_negotiation.cc:407] Connection header received
0808 07:11:12.700563 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.700565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.700577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.700635 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.701272 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.701778 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.702804 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.703006 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.703090 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.705247 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.705249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.705249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.705256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.706463 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.706991 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.706992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.706993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.707057 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.707361 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.707363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.707363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.707505 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.707604 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.707906 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.707994 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:12.708298 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.699245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34045 (local address 127.22.2.131:54555)
0808 07:11:12.699433 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.699469 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.700447 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.700709 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.700713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.700982 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:12.701170 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.701174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.701874 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.701875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.702692 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.702694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.703103 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.703104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.703237 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.703787 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:12.703795 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:12.705154 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:12.706539 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:12.706546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:12.706548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:12.706870 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:12.707130 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:12.707130 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:12.707131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:12.707285 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:12.707613 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:12.707621 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:12.707765 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.707881 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.708043 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260808 07:11:12.708709 23666 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34045
I20260808 07:11:12.708796 23666 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:12.708992 23666 heartbeater.cc:511] Master 127.22.2.190:34045 requested a full tablet report, sending...
I20260808 07:11:12.709355 23162 ts_manager.cc:194] Registered new tserver with Master: 2e7a58316aed4f51aee779a9738c6f29 (127.22.2.131:39063)
I20260808 07:11:12.709725 23162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:54555
I20260808 07:11:12.718788 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:11:12.726698 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.720498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.22.2.190:34045)
0808 07:11:12.720572 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.720575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.720670 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:11:12.720742 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.720743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.720754 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.720827 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.721398 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.721906 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.722703 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.722904 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.722978 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.724259 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.724261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.724261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.724268 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.725485 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.725840 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.725841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.725841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.725905 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.726123 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.726124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.726125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.726276 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.726381 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.726513 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.726595 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:11:12.728971 23162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:12.730558 23162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:12.737450 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.733020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:39063 (local address 127.0.0.1:33244)
0808 07:11:12.733910 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.733919 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.733965 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.734167 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.734167 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.734170 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.734221 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.734222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.735105 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.735106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.736283 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.736286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.737178 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.737179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.737234 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.737246 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.737303 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.737370 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":23,"thread_start_us":88,"threads_started":1}
I20260808 07:11:12.737463 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.733245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33244 (local address 127.22.2.131:39063)
0808 07:11:12.733440 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.733444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.734028 (+ 584us) server_negotiation.cc:407] Connection header received
0808 07:11:12.734078 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.734080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.734097 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.734170 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.734286 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.734987 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.736377 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.737093 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.737174 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.737235 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.737318 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.737367 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260808 07:11:12.737970 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.733191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.22.2.129:33395)
0808 07:11:12.733426 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.733430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.733438 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:12.733469 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.733471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.733483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.733572 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.733766 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.734943 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.736587 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.737292 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.737363 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.737414 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.737859 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.737906 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":113,"threads_started":1}
I20260808 07:11:12.738214 23678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.732998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33395 (local address 127.0.0.1:51964)
0808 07:11:12.733161 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.733200 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.733265 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.733577 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.733581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.733584 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.733666 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.733669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.735058 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.735059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.736509 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.736511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.737382 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.737382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.737440 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.737444 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.737496 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.738145 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":35,"thread_start_us":81,"threads_started":1}
I20260808 07:11:12.738294 23593 tablet_service.cc:1467] Processing CreateTablet for tablet 4c616340eba24bcf855a5aba1aa44f00 (DEFAULT_TABLE table=test-table [id=94bd1110f92943afb8e63beff6699710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.738538 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.732998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35599 (local address 127.0.0.1:34466)
0808 07:11:12.733736 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.733747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.733805 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.734088 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.734089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.734091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.734131 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.734133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.735381 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.735382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.736496 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.736499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.738285 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.738286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.738367 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.738371 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.738425 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.738465 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260808 07:11:12.738734 23593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c616340eba24bcf855a5aba1aa44f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.738790 23682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.733222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.22.2.130:35599)
0808 07:11:12.733411 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.733415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.733822 (+ 407us) server_negotiation.cc:407] Connection header received
0808 07:11:12.733897 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.733900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.733915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.733998 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.734207 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.735279 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.737452 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.738108 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.738180 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.738246 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.738646 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.738697 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
I20260808 07:11:12.738989 23308 tablet_service.cc:1467] Processing CreateTablet for tablet 4c616340eba24bcf855a5aba1aa44f00 (DEFAULT_TABLE table=test-table [id=94bd1110f92943afb8e63beff6699710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.739254 23308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c616340eba24bcf855a5aba1aa44f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.741597 23685 tablet_bootstrap.cc:492] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29: Bootstrap starting.
I20260808 07:11:12.742063 23686 tablet_bootstrap.cc:492] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07: Bootstrap starting.
I20260808 07:11:12.742136 23450 tablet_service.cc:1467] Processing CreateTablet for tablet 4c616340eba24bcf855a5aba1aa44f00 (DEFAULT_TABLE table=test-table [id=94bd1110f92943afb8e63beff6699710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.742306 23685 tablet_bootstrap.cc:654] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.742475 23450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c616340eba24bcf855a5aba1aa44f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.742825 23686 tablet_bootstrap.cc:654] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.743000 23685 log.cc:824] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:12.743479 23686 log.cc:824] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:12.743851 23685 tablet_bootstrap.cc:492] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29: No bootstrap required, opened a new log
I20260808 07:11:12.743899 23685 ts_tablet_manager.cc:1397] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:12.744309 23686 tablet_bootstrap.cc:492] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07: No bootstrap required, opened a new log
I20260808 07:11:12.744367 23686 ts_tablet_manager.cc:1397] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:11:12.745409 23690 tablet_bootstrap.cc:492] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0: Bootstrap starting.
I20260808 07:11:12.746013 23690 tablet_bootstrap.cc:654] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.745987 23685 raft_consensus.cc:348] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.746182 23685 raft_consensus.cc:374] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.746222 23685 raft_consensus.cc:729] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e7a58316aed4f51aee779a9738c6f29, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.746359 23685 consensus_queue.cc:260] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.746551 23690 log.cc:824] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:12.746644 23685 ts_tablet_manager.cc:1428] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:12.746599 23686 raft_consensus.cc:348] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.746762 23686 raft_consensus.cc:374] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.746802 23686 raft_consensus.cc:729] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af08ffd8991542a7af4b97815d3c8e07, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.746943 23686 consensus_queue.cc:260] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.747228 23666 heartbeater.cc:503] Master 127.22.2.190:34045 was elected leader, sending a full tablet report...
I20260808 07:11:12.747215 23686 ts_tablet_manager.cc:1428] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:12.747272 23378 heartbeater.cc:503] Master 127.22.2.190:34045 was elected leader, sending a full tablet report...
I20260808 07:11:12.747411 23690 tablet_bootstrap.cc:492] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0: No bootstrap required, opened a new log
I20260808 07:11:12.747465 23690 ts_tablet_manager.cc:1397] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:12.749157 23690 raft_consensus.cc:348] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.749293 23690 raft_consensus.cc:374] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.749316 23690 raft_consensus.cc:729] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2630df56a74bd3b86904fb3ef8ffb0, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.749388 23690 consensus_queue.cc:260] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.749571 23690 ts_tablet_manager.cc:1428] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:11:12.749627 23518 heartbeater.cc:503] Master 127.22.2.190:34045 was elected leader, sending a full tablet report...
I20260808 07:11:12.783859 23694 raft_consensus.cc:482] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:12.783998 23694 raft_consensus.cc:504] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.784371 23694 leader_election.cc:288] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af08ffd8991542a7af4b97815d3c8e07 (127.22.2.129:33395), 2e7a58316aed4f51aee779a9738c6f29 (127.22.2.131:39063)
W20260808 07:11:12.786244 23519 tablet.cc:2369] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:12.788306 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.784623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:41337 (local address 127.22.2.129:33395)
0808 07:11:12.784671 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.784673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.784684 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:12.784747 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.784747 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.784757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.784809 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.784964 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.785735 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.787268 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.787919 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.787999 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.788009 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.788194 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.788234 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:12.788321 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.784623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:39645 (local address 127.22.2.131:39063)
0808 07:11:12.784671 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.784674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.784873 (+ 199us) server_negotiation.cc:407] Connection header received
0808 07:11:12.784910 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.784911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.784922 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.784974 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.785108 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.785787 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.787289 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.787919 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.788001 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.788009 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.788194 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.788234 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:12.788338 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.784570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33395 (local address 127.22.2.130:41337)
0808 07:11:12.784629 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.784641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.784685 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.784822 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.784823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.784825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.784902 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.784905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.785835 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.785836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.787162 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.787165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.788014 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.788015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.788114 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.788116 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.788173 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.788223 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":52}
I20260808 07:11:12.788347 23695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.784570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:39063 (local address 127.22.2.130:39645)
0808 07:11:12.784782 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.784792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.784846 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.784984 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.784984 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.784986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.785049 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.785049 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.785856 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.785856 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.787162 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.787165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.788014 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.788015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.788111 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.788116 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.788172 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.788223 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":79,"thread_start_us":116,"threads_started":1}
I20260808 07:11:12.788813 23328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c616340eba24bcf855a5aba1aa44f00" candidate_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af08ffd8991542a7af4b97815d3c8e07" is_pre_election: true
I20260808 07:11:12.788986 23328 raft_consensus.cc:2454] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f2630df56a74bd3b86904fb3ef8ffb0 in term 0.
I20260808 07:11:12.789256 23402 leader_election.cc:302] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f2630df56a74bd3b86904fb3ef8ffb0, af08ffd8991542a7af4b97815d3c8e07; no voters:
I20260808 07:11:12.789283 23615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c616340eba24bcf855a5aba1aa44f00" candidate_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e7a58316aed4f51aee779a9738c6f29" is_pre_election: true
I20260808 07:11:12.789419 23615 raft_consensus.cc:2454] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f2630df56a74bd3b86904fb3ef8ffb0 in term 0.
I20260808 07:11:12.789469 23694 raft_consensus.cc:2781] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:12.789538 23694 raft_consensus.cc:482] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:12.789574 23694 raft_consensus.cc:3037] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.790455 23694 raft_consensus.cc:504] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.790613 23694 leader_election.cc:288] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [CANDIDATE]: Term 1 election: Requested vote from peers af08ffd8991542a7af4b97815d3c8e07 (127.22.2.129:33395), 2e7a58316aed4f51aee779a9738c6f29 (127.22.2.131:39063)
I20260808 07:11:12.790817 23328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c616340eba24bcf855a5aba1aa44f00" candidate_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af08ffd8991542a7af4b97815d3c8e07"
I20260808 07:11:12.790899 23328 raft_consensus.cc:3037] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.790879 23615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c616340eba24bcf855a5aba1aa44f00" candidate_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e7a58316aed4f51aee779a9738c6f29"
I20260808 07:11:12.790987 23615 raft_consensus.cc:3037] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.791702 23328 raft_consensus.cc:2454] T 4c616340eba24bcf855a5aba1aa44f00 P af08ffd8991542a7af4b97815d3c8e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f2630df56a74bd3b86904fb3ef8ffb0 in term 1.
I20260808 07:11:12.791867 23402 leader_election.cc:302] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f2630df56a74bd3b86904fb3ef8ffb0, af08ffd8991542a7af4b97815d3c8e07; no voters:
I20260808 07:11:12.791872 23615 raft_consensus.cc:2454] T 4c616340eba24bcf855a5aba1aa44f00 P 2e7a58316aed4f51aee779a9738c6f29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f2630df56a74bd3b86904fb3ef8ffb0 in term 1.
I20260808 07:11:12.791952 23694 raft_consensus.cc:2781] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:12.792136 23694 raft_consensus.cc:686] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [term 1 LEADER]: Becoming Leader. State: Replica: 6f2630df56a74bd3b86904fb3ef8ffb0, State: Running, Role: LEADER
I20260808 07:11:12.792227 23694 consensus_queue.cc:237] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } }
I20260808 07:11:12.793118 23162 catalog_manager.cc:5495] T 4c616340eba24bcf855a5aba1aa44f00 P 6f2630df56a74bd3b86904fb3ef8ffb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f2630df56a74bd3b86904fb3ef8ffb0 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af08ffd8991542a7af4b97815d3c8e07" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f2630df56a74bd3b86904fb3ef8ffb0" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e7a58316aed4f51aee779a9738c6f29" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39063 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:12.816550 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.809002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.22.2.129:33395)
0808 07:11:12.809072 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.809074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.809239 (+ 165us) server_negotiation.cc:407] Connection header received
0808 07:11:12.809311 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.809312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.809322 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.809359 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.809970 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.810433 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.811131 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.811326 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.811381 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.813828 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.813840 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.813840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.813846 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.815505 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.815873 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.815876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.815876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.815934 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.816081 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.816081 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.816081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.816206 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.816307 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.816367 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.816427 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:12.840763 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.833649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.22.2.129:33395)
0808 07:11:12.833755 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.833757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.833847 (+ 90us) server_negotiation.cc:407] Connection header received
0808 07:11:12.833903 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.833904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.833914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.833967 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.834492 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.834982 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.835754 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.835938 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.835994 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.838133 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.838134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.838134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.838141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.839535 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.840004 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.840004 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.840005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.840057 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.840281 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.840282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.840282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.840375 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.840446 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.840632 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.840693 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:12.864027 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.856661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.22.2.129:33395)
0808 07:11:12.856756 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.856759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.856878 (+ 119us) server_negotiation.cc:407] Connection header received
0808 07:11:12.856983 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.856984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.856995 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.857048 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.857617 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.858123 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.858958 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.859146 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.859218 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.861442 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.861444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.861444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.861451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.862797 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.863185 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.863187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.863187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.863250 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.863460 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.863461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.863461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.863605 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.863703 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.863839 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.863917 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 07:11:12.864648 23286 tablet_service.cc:2908] Rejecting scan request for tablet 4c616340eba24bcf855a5aba1aa44f00: Uninitialized: clean time has not yet been initialized
I20260808 07:11:12.865486 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 23237
I20260808 07:11:12.873070 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 23384
I20260808 07:11:12.879369 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 23524
I20260808 07:11:12.884881 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 23123
2026-08-08T07:11:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2127 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23712](info): set up 2 sockets
Aug 08 07:11:12 dist-test-slave-6k22 krb5kdc[23712](info): commencing operation
krb5kdc: starting...
W20260808 07:11:13.995123 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:14.122555 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34327
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:34327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:14.208750 23726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:14.208987 23726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:14.209026 23726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:14.210198 23726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:14.210268 23726 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:11:14.210290 23726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:14.210314 23726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:14.210367 23726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:14.210400 23726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:14.211937 23726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:34327
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:14.212229 23726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:14.212463 23726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:14.217512 23734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:14.217499 23736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:14.217489 23733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:14.217742 23726 server_base.cc:1034] running on GCE node
I20260808 07:11:14.217979 23726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:14.218217 23726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:14.219404 23726 hybrid_clock.cc:648] HybridClock initialized: now 1786173074219390 us; error 41 us; skew 500 ppm
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:14.222249 23726 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:14.222671 23726 webserver.cc:460] Webserver started at http://127.22.2.190:45027/ using document root <none> and password file <none>
I20260808 07:11:14.222893 23726 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:14.222939 23726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:14.223100 23726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:14.224334 23726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "613744e19d484e31a2baf77e1c854572"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "0e8eaa08f015015e7b6b9dff0c00fec5"
server_key_iv: "c97903ebb7c5c4489980fdba3fd03da0"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.224697 23726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "613744e19d484e31a2baf77e1c854572"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "0e8eaa08f015015e7b6b9dff0c00fec5"
server_key_iv: "c97903ebb7c5c4489980fdba3fd03da0"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.226046 23726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:14.226959 23743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.227210 23726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:14.227327 23726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "613744e19d484e31a2baf77e1c854572"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "0e8eaa08f015015e7b6b9dff0c00fec5"
server_key_iv: "c97903ebb7c5c4489980fdba3fd03da0"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.227407 23726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:14.235680 23726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:14.237825 23726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:14.237984 23726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:14.243191 23726 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:34327
I20260808 07:11:14.243235 23813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:34327 every 8 connection(s)
I20260808 07:11:14.243705 23726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:14.244386 23814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:14.246588 23814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572: Bootstrap starting.
I20260808 07:11:14.247128 23814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:14.247434 23814 log.cc:824] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:14.247601 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 23726
I20260808 07:11:14.247707 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:14.247936 22538 external_mini_cluster.cc:1428] Setting key 24a48022da3f2b745141b7d5262ad4ef
I20260808 07:11:14.248276 23814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572: No bootstrap required, opened a new log
I20260808 07:11:14.250638 23814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613744e19d484e31a2baf77e1c854572" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34327 } }
I20260808 07:11:14.250829 23814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:14.250875 23814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 613744e19d484e31a2baf77e1c854572, State: Initialized, Role: FOLLOWER
I20260808 07:11:14.251024 23814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613744e19d484e31a2baf77e1c854572" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34327 } }
I20260808 07:11:14.251113 23814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:14.251148 23814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:14.251202 23814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:14.251916 23814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613744e19d484e31a2baf77e1c854572" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34327 } }
I20260808 07:11:14.252012 23814 leader_election.cc:302] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 613744e19d484e31a2baf77e1c854572; no voters:
I20260808 07:11:14.252250 23814 leader_election.cc:288] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:14.252277 23819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:14.252393 23819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [term 1 LEADER]: Becoming Leader. State: Replica: 613744e19d484e31a2baf77e1c854572, State: Running, Role: LEADER
I20260808 07:11:14.252542 23814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:14.252552 23819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613744e19d484e31a2baf77e1c854572" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34327 } }
I20260808 07:11:14.253026 23821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "613744e19d484e31a2baf77e1c854572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613744e19d484e31a2baf77e1c854572" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34327 } } }
I20260808 07:11:14.253309 23821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:14.253226 23820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 613744e19d484e31a2baf77e1c854572. Latest consensus state: current_term: 1 leader_uuid: "613744e19d484e31a2baf77e1c854572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613744e19d484e31a2baf77e1c854572" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34327 } } }
I20260808 07:11:14.253679 23820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572 [sys.catalog]: This master's current role is: LEADER
W20260808 07:11:14.254539 23835 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:14.255156 23835 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:14.255234 23837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:14.255419 23837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:14.260051 23837 catalog_manager.cc:1347] Generated new cluster ID: dce1531e9c9041b1aa39716b52ee1f80
I20260808 07:11:14.260108 23837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:14.260704 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.248460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.22.2.190:34327)
0808 07:11:14.248662 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.248671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.248695 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:11:14.250610 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.250617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.250664 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.250780 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:14.251640 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.252787 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.253833 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.254399 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.254519 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.257312 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:14.257315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:14.257316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:14.257326 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:14.258986 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.259409 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.259411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.259412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.259526 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.259827 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.259828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.259828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.260144 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:14.260314 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.260383 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.260525 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
I20260808 07:11:14.270896 23837 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:14.271449 23837 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:14.276772 23837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572: Generated new TSK 0
I20260808 07:11:14.276968 23837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:14.311532 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34327
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:14.399551 23842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:14.399916 23842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:14.399945 23842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:14.401180 23842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:14.401228 23842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:14.401249 23842 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:14.401291 23842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:14.402771 23842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:14.403038 23842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:14.403285 23842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:14.408455 23849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:14.408443 23847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:14.408607 23842 server_base.cc:1034] running on GCE node
W20260808 07:11:14.408443 23852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:14.409032 23842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:14.409312 23842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:14.410476 23842 hybrid_clock.cc:648] HybridClock initialized: now 1786173074410463 us; error 26 us; skew 500 ppm
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:14.413069 23842 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:14.413496 23842 webserver.cc:460] Webserver started at http://127.22.2.129:40953/ using document root <none> and password file <none>
I20260808 07:11:14.413813 23842 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:14.413902 23842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:14.414063 23842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:14.415158 23842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "6b4f319d65c6405ba0b0e7924e3609b5"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "ca0aaac9ccde05a6f53aaae5822275b2"
server_key_iv: "3851905f950ada430c26386a6deac705"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.415535 23842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "6b4f319d65c6405ba0b0e7924e3609b5"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "ca0aaac9ccde05a6f53aaae5822275b2"
server_key_iv: "3851905f950ada430c26386a6deac705"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.416939 23842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:14.417753 23861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.417938 23842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:14.418040 23842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "6b4f319d65c6405ba0b0e7924e3609b5"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "ca0aaac9ccde05a6f53aaae5822275b2"
server_key_iv: "3851905f950ada430c26386a6deac705"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.418128 23842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:14.449012 23842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:14.451864 23842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:14.452020 23842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:14.452669 23842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:14.452739 23842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.452769 23842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:14.452867 23842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:14.461854 23842 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33775
I20260808 07:11:14.461997 23989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33775 every 8 connection(s)
I20260808 07:11:14.462374 23842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:14.464774 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.453085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:57827 (local address 127.22.2.190:34327)
0808 07:11:14.453146 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.453149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.453411 (+ 262us) server_negotiation.cc:407] Connection header received
0808 07:11:14.454014 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.454016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.454037 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.454139 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:14.455047 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.456041 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.457131 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.457462 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.457531 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.461478 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:14.461480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:14.461481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:14.461492 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:14.463231 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.463877 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.463878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.463879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.463934 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.464156 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.464156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.464156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.464279 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:14.464331 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.464631 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.464704 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:11:14.465071 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.453088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34327 (local address 127.22.2.129:57827)
0808 07:11:14.453271 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.453332 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.453894 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.454248 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.454251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.454594 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:14.454922 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.454929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.456164 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.456165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.457027 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.457030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.457605 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.457606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.457791 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.458442 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:14.458452 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:14.461355 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:14.463375 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.463385 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.463387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.463799 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.463993 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.463994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.463994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.464098 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.464368 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:14.464375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:14.464500 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.464609 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.464791 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260808 07:11:14.465662 23990 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34327
I20260808 07:11:14.465760 23990 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:14.465955 23990 heartbeater.cc:511] Master 127.22.2.190:34327 requested a full tablet report, sending...
I20260808 07:11:14.466576 23762 ts_manager.cc:194] Registered new tserver with Master: 6b4f319d65c6405ba0b0e7924e3609b5 (127.22.2.129:33775)
I20260808 07:11:14.467511 23762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:57827
I20260808 07:11:14.468463 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 23842
I20260808 07:11:14.468580 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:14.468765 22538 external_mini_cluster.cc:1428] Setting key e02080e3e6f42f8cdf1080cfa8085f98
I20260808 07:11:14.473232 23996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.466709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34327 (local address 127.22.2.129:37797)
0808 07:11:14.466848 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.466859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.466924 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.467046 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.467047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.467246 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:14.467312 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.467313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.468325 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.468326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.468899 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.468901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.469369 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.469370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.469457 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.469916 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:14.469921 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:14.470201 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:14.472074 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.472075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.472076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.472254 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.472500 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.472501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.472501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.472577 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.472926 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:14.472927 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:14.472976 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.473051 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.473154 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260808 07:11:14.473323 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.466716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:37797 (local address 127.22.2.190:34327)
0808 07:11:14.466762 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.466764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.466912 (+ 148us) server_negotiation.cc:407] Connection header received
0808 07:11:14.466981 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.466981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.466991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.467033 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:14.467374 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.468227 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.468999 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.469291 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.469358 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.470320 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:14.470321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:14.470322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:14.470328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:14.471985 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.472329 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.472330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.472331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.472391 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.472699 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.472700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.472701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.472850 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:14.472921 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.473084 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.473176 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:14.495986 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34327
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:14.579504 23999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:14.579779 23999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:14.579809 23999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:14.581051 23999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:14.581094 23999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:14.581115 23999 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:14.581132 23999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:14.582646 23999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:14.582866 23999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:14.583050 23999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:14.588181 24005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:14.588182 24008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:14.588181 24006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:14.588371 23999 server_base.cc:1034] running on GCE node
I20260808 07:11:14.588761 23999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:14.589022 23999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:14.590193 23999 hybrid_clock.cc:648] HybridClock initialized: now 1786173074590185 us; error 39 us; skew 500 ppm
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:14.592859 23999 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:14.593310 23999 webserver.cc:460] Webserver started at http://127.22.2.130:37093/ using document root <none> and password file <none>
I20260808 07:11:14.593554 23999 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:14.593623 23999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:14.593780 23999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:14.594842 23999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "9caf0e680d284690bed4d39e7c38c098"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "7cf5f2e01bc1ea6c717c72023bf2af29"
server_key_iv: "a821f24598526c35afb91d674e4e3ea3"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.595252 23999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "9caf0e680d284690bed4d39e7c38c098"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "7cf5f2e01bc1ea6c717c72023bf2af29"
server_key_iv: "a821f24598526c35afb91d674e4e3ea3"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.596736 23999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:14.597500 24016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.597673 23999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.597776 23999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "9caf0e680d284690bed4d39e7c38c098"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "7cf5f2e01bc1ea6c717c72023bf2af29"
server_key_iv: "a821f24598526c35afb91d674e4e3ea3"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.597858 23999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:14.608661 23999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:14.610993 23999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:14.611161 23999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:14.611778 23999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:14.611842 23999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.611886 23999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:14.611912 23999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.618933 23999 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35757
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:14.619133 24151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35757 every 8 connection(s)
I20260808 07:11:14.619400 23999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:14.621269 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 23999
I20260808 07:11:14.621348 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:14.621526 22538 external_mini_cluster.cc:1428] Setting key 56dfd8ca31ebc0465b56582811d88503
I20260808 07:11:14.623747 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.612251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:57691 (local address 127.22.2.190:34327)
0808 07:11:14.612296 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.612303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.612529 (+ 226us) server_negotiation.cc:407] Connection header received
0808 07:11:14.613088 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.613090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.613104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.613155 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:14.613845 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.614658 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.616352 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.616709 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.616793 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.619987 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:14.619989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:14.619990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:14.619999 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:14.622097 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.622597 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.622599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.622600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.622679 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.622925 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.622926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.622927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.623121 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:14.623210 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.623468 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.623596 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:11:14.623801 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.612242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34327 (local address 127.22.2.130:57691)
0808 07:11:14.612411 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.612445 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.612974 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.613220 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.613224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.613541 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:14.613730 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.613760 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.614776 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.614779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.616268 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.616273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.616829 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.616830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.616994 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.617490 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:14.617501 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:14.619840 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:14.622237 (+ 2397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.622258 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.622260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.622512 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.622758 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.622758 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.622759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.622855 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.623200 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:14.623205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:14.623330 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.623412 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.623532 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260808 07:11:14.625573 24152 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34327
I20260808 07:11:14.625689 24152 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:14.625882 24152 heartbeater.cc:511] Master 127.22.2.190:34327 requested a full tablet report, sending...
I20260808 07:11:14.626214 23762 ts_manager.cc:194] Registered new tserver with Master: 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757)
I20260808 07:11:14.626595 23762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:57691
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 07:11:14.633445 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.625778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:49245 (local address 127.22.2.190:34327)
0808 07:11:14.625859 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.625861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.626206 (+ 345us) server_negotiation.cc:407] Connection header received
0808 07:11:14.626325 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.626327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.626340 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.626377 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:14.627719 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.628177 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.629406 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.629575 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.629624 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.630802 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:14.630803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:14.630803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:14.630809 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:14.632233 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.632705 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.632706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.632706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.632766 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.633003 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.633003 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.633003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.633135 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:14.633184 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.633329 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.633385 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 07:11:14.633473 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.625670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34327 (local address 127.22.2.130:49245)
0808 07:11:14.626124 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.626138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.626238 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.626451 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.626452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.626732 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:14.626825 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.626828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.628295 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.628296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.629311 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.629315 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.629658 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.629659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.629753 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.630174 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:14.630181 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:14.630695 (+ 514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:14.632322 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.632324 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.632324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.632609 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.632842 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.632843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.632843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.632924 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.633198 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:14.633199 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:14.633240 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.633312 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.633402 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":76,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:14.650606 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34327
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:14.743265 24162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:14.743546 24162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:14.743609 24162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:14.745028 24162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:14.745117 24162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:14.745159 24162 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:14.745216 24162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:14.746860 24162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:14.747170 24162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:14.747434 24162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:14.753024 24171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:14.753054 24168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:14.753047 24169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:14.753321 24162 server_base.cc:1034] running on GCE node
I20260808 07:11:14.753561 24162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:14.753863 24162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:14.755067 24162 hybrid_clock.cc:648] HybridClock initialized: now 1786173074755050 us; error 30 us; skew 500 ppm
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:14.757566 24162 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:14.758059 24162 webserver.cc:460] Webserver started at http://127.22.2.131:44363/ using document root <none> and password file <none>
I20260808 07:11:14.758325 24162 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:14.758407 24162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:14.758556 24162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:14.759670 24162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "fc8b878824a0bb9295a7c575b1ca310f"
server_key_iv: "18eb17d85a131805522ad49706c5c1c5"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.760030 24162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "fc8b878824a0bb9295a7c575b1ca310f"
server_key_iv: "18eb17d85a131805522ad49706c5c1c5"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.761368 24162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:14.762151 24179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.762346 24162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:14.762449 24162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e"
format_stamp: "Formatted at 2026-08-08 07:11:14 on dist-test-slave-6k22"
server_key: "fc8b878824a0bb9295a7c575b1ca310f"
server_key_iv: "18eb17d85a131805522ad49706c5c1c5"
server_key_version: "encryptionkey@0"
I20260808 07:11:14.762521 24162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:14.778515 24162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:14.780668 24162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:14.780831 24162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:14.781424 24162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:14.781491 24162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:14.781541 24162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:14.781565 24162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:14.789453 24162 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:37325
I20260808 07:11:14.789536 24306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:37325 every 8 connection(s)
I20260808 07:11:14.789902 24162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:14.792124 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.781819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:33365 (local address 127.22.2.190:34327)
0808 07:11:14.781875 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.781878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.782150 (+ 272us) server_negotiation.cc:407] Connection header received
0808 07:11:14.782939 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.782940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.782951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.783032 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:14.784023 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.784604 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.786324 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.786565 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.786729 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.789358 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:14.789368 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:14.789368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:14.789375 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:14.790912 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.791292 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.791293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.791293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.791346 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.791509 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.791509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.791509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.791693 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:14.791747 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.791991 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.792057 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:11:14.792273 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.781801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34327 (local address 127.22.2.131:33365)
0808 07:11:14.781988 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.782040 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.782815 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.783132 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.783137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.783592 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:14.783900 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.783908 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.784745 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.784747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.786234 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.786238 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.786677 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.786678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.786865 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.787469 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:14.787481 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:14.789246 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:14.790992 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.790998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.791000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.791225 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.791388 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.791389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.791389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.791473 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.791753 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:14.791758 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:14.791874 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.791962 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.792081 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260808 07:11:14.792827 24307 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34327
I20260808 07:11:14.792913 24307 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:14.793092 24307 heartbeater.cc:511] Master 127.22.2.190:34327 requested a full tablet report, sending...
I20260808 07:11:14.793475 23760 ts_manager.cc:194] Registered new tserver with Master: ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325)
I20260808 07:11:14.794011 23760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:33365
I20260808 07:11:14.796702 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 24162
I20260808 07:11:14.796804 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:14.796984 22538 external_mini_cluster.cc:1428] Setting key d6a1ada20e8a91b8bf8def5f9be01b25
I20260808 07:11:14.797915 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:11:14.799844 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.793901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:46743 (local address 127.22.2.190:34327)
0808 07:11:14.793943 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.793945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.794017 (+ 72us) server_negotiation.cc:407] Connection header received
0808 07:11:14.794098 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.794098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.794109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.794160 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:14.794498 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.794955 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.795744 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.795949 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.796034 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.797052 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:14.797053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:14.797054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:14.797062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:14.798650 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.799002 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.799003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.799004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.799061 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.799272 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.799273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.799274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.799389 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:14.799467 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.799675 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.799753 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:14.799849 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.793810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34327 (local address 127.22.2.131:46743)
0808 07:11:14.793956 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.793967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.794028 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.794177 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.794178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.794359 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:14.794412 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.794413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.795047 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.795048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.795613 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.795614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.796037 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.796038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.796116 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.796557 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:14.796562 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:14.796956 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:14.798754 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.798755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.798756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.798928 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.799136 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:14.799137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:14.799137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:14.799198 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:14.799467 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:14.799468 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:14.799514 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.799648 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.799756 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:11:14 dist-test-slave-6k22 krb5kdc[23712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:14.823549 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.815744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.22.2.190:34327)
0808 07:11:14.815849 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.815851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.815865 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:11:14.815943 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.815944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.815956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.816024 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:14.816623 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.817167 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.817921 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.818128 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.818194 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.820711 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:14.820712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:14.820713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:14.820721 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:14.822239 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.822587 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.822590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.822591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.822672 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:14.822884 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:14.822885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:14.822885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:14.823056 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:14.823174 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.823345 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.823438 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:11:14.825747 23760 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:14.827279 23760 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:14.834296 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.829691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33775 (local address 127.0.0.1:46058)
0808 07:11:14.830396 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.830406 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.830454 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.830905 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.830905 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.830907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:14.830937 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.830938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.831951 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.831952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.833086 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.833089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.833952 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.833953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.834031 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.834046 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.834126 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.834187 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":36,"thread_start_us":101,"threads_started":1}
I20260808 07:11:14.834331 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.829988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.22.2.129:33775)
0808 07:11:14.830314 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.830319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.830514 (+ 195us) server_negotiation.cc:407] Connection header received
0808 07:11:14.830562 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.830564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.830580 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.830655 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:14.831047 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.831865 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.833183 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.833851 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.833929 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.833989 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.834158 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.834210 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":92,"threads_started":1}
I20260808 07:11:14.834703 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.829686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:37325 (local address 127.0.0.1:33664)
0808 07:11:14.830119 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.830169 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.830262 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.830763 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.830764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.830765 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:14.830803 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.830804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.832236 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.832238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.833432 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.833435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.834442 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.834444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.834513 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.834517 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.834578 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.834618 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":2,"thread_start_us":124,"threads_started":1}
I20260808 07:11:14.834736 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.829961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.22.2.131:37325)
0808 07:11:14.830220 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.830225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.830238 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:11:14.830386 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.830389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.830410 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.830490 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:14.830928 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.832142 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.833527 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.834272 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.834345 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.834402 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.834596 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.834637 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":123,"threads_started":1}
I20260808 07:11:14.835171 23916 tablet_service.cc:1467] Processing CreateTablet for tablet 0c19987395e641f7a1514215087e8ea7 (DEFAULT_TABLE table=test-table [id=d5d91f98dd1e41609aa10bfe0bc5910d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:14.835322 24235 tablet_service.cc:1467] Processing CreateTablet for tablet 0c19987395e641f7a1514215087e8ea7 (DEFAULT_TABLE table=test-table [id=d5d91f98dd1e41609aa10bfe0bc5910d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:14.835479 23916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c19987395e641f7a1514215087e8ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:14.835640 24235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c19987395e641f7a1514215087e8ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:14.836012 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.829686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35757 (local address 127.0.0.1:55322)
0808 07:11:14.830187 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:14.830198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:14.830282 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:14.830721 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:14.830724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:14.830726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:14.830803 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.830804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.832230 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.832231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.833435 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:14.833437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.834281 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.834282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:14.834356 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.835816 (+ 1460us) client_negotiation.cc:769] Sending connection context
0808 07:11:14.835859 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:11:14.835908 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":84,"thread_start_us":67,"threads_started":1}
I20260808 07:11:14.836535 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:14.829909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55322 (local address 127.22.2.130:35757)
0808 07:11:14.830143 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:14.830149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:14.830270 (+ 121us) server_negotiation.cc:407] Connection header received
0808 07:11:14.830598 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:14.830601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:14.830617 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:14.830692 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:14.830913 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.832124 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.833558 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:14.834179 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:14.836282 (+ 2103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:14.836362 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:14.836390 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 07:11:14.836434 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":95,"threads_started":1}
I20260808 07:11:14.838730 24331 tablet_bootstrap.cc:492] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Bootstrap starting.
I20260808 07:11:14.839367 24332 tablet_bootstrap.cc:492] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Bootstrap starting.
I20260808 07:11:14.839659 24078 tablet_service.cc:1467] Processing CreateTablet for tablet 0c19987395e641f7a1514215087e8ea7 (DEFAULT_TABLE table=test-table [id=d5d91f98dd1e41609aa10bfe0bc5910d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:14.839867 24331 tablet_bootstrap.cc:654] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:14.840018 24078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c19987395e641f7a1514215087e8ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:14.840068 24332 tablet_bootstrap.cc:654] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:14.840240 24331 log.cc:824] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:14.840410 24332 log.cc:824] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:14.841085 24331 tablet_bootstrap.cc:492] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: No bootstrap required, opened a new log
I20260808 07:11:14.841095 24332 tablet_bootstrap.cc:492] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: No bootstrap required, opened a new log
I20260808 07:11:14.841166 24331 ts_tablet_manager.cc:1397] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:11:14.841185 24332 ts_tablet_manager.cc:1397] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:14.842711 24332 raft_consensus.cc:348] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:14.842876 24332 raft_consensus.cc:374] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:14.842922 24332 raft_consensus.cc:729] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4f319d65c6405ba0b0e7924e3609b5, State: Initialized, Role: FOLLOWER
I20260808 07:11:14.843063 24332 consensus_queue.cc:260] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:14.843369 24332 ts_tablet_manager.cc:1428] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:14.843343 24331 raft_consensus.cc:348] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:14.843510 23990 heartbeater.cc:503] Master 127.22.2.190:34327 was elected leader, sending a full tablet report...
I20260808 07:11:14.843515 24331 raft_consensus.cc:374] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:14.843590 24331 raft_consensus.cc:729] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce73d7c9195a4ce1b7b795f9eca6dd8e, State: Initialized, Role: FOLLOWER
I20260808 07:11:14.843706 24331 consensus_queue.cc:260] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:14.844008 24307 heartbeater.cc:503] Master 127.22.2.190:34327 was elected leader, sending a full tablet report...
I20260808 07:11:14.844178 24336 tablet_bootstrap.cc:492] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Bootstrap starting.
I20260808 07:11:14.843961 24331 ts_tablet_manager.cc:1428] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260808 07:11:14.844902 24336 tablet_bootstrap.cc:654] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:14.845232 24336 log.cc:824] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:14.846051 24336 tablet_bootstrap.cc:492] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: No bootstrap required, opened a new log
I20260808 07:11:14.846108 24336 ts_tablet_manager.cc:1397] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:14.847455 24336 raft_consensus.cc:348] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:14.847657 24336 raft_consensus.cc:374] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:14.847692 24336 raft_consensus.cc:729] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caf0e680d284690bed4d39e7c38c098, State: Initialized, Role: FOLLOWER
I20260808 07:11:14.847786 24336 consensus_queue.cc:260] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:14.847983 24336 ts_tablet_manager.cc:1428] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:14.848057 24152 heartbeater.cc:503] Master 127.22.2.190:34327 was elected leader, sending a full tablet report...
W20260808 07:11:14.870195 24153 tablet.cc:2369] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:14.963296 23991 tablet.cc:2369] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:15.040840 24308 tablet.cc:2369] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:15.054418 24335 raft_consensus.cc:482] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:15.054558 24335 raft_consensus.cc:504] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:15.054936 24335 leader_election.cc:288] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757), ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325)
I20260808 07:11:15.059130 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.055232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:42787 (local address 127.22.2.130:35757)
0808 07:11:15.055289 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.055291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.055379 (+ 88us) server_negotiation.cc:407] Connection header received
0808 07:11:15.055478 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.055480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.055494 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.055582 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:15.055781 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.056555 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.058099 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.058698 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.058777 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.058786 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.059003 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.059059 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:11:15.059130 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.055314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:57453 (local address 127.22.2.131:37325)
0808 07:11:15.055370 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.055372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.055435 (+ 63us) server_negotiation.cc:407] Connection header received
0808 07:11:15.055496 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.055497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.055505 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.055593 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:15.055777 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.056556 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.058098 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.058698 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.058778 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.058786 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.059003 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.059054 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:11:15.059151 24341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.055141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:37325 (local address 127.22.2.129:57453)
0808 07:11:15.055369 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:15.055378 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:15.055431 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:15.055601 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:15.055602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:15.055603 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:15.055702 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.055704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.056645 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.056646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.057996 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.058000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.058795 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.058797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.058891 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.058898 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:11:15.058960 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:11:15.059016 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":85,"thread_start_us":66,"threads_started":1}
I20260808 07:11:15.059151 24342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.055141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35757 (local address 127.22.2.129:42787)
0808 07:11:15.055290 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:15.055301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:15.055394 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:15.055601 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:15.055602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:15.055606 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:15.055687 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.055690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.056646 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.056646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.057996 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.058000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.058797 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.058798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.058891 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.058898 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:11:15.058960 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:11:15.059016 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":78,"thread_start_us":66,"threads_started":1}
I20260808 07:11:15.059414 24338 raft_consensus.cc:482] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:15.059518 24338 raft_consensus.cc:504] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:15.059715 24258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c19987395e641f7a1514215087e8ea7" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" is_pre_election: true
I20260808 07:11:15.059841 24258 raft_consensus.cc:2454] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 0.
I20260808 07:11:15.059819 24101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c19987395e641f7a1514215087e8ea7" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caf0e680d284690bed4d39e7c38c098" is_pre_election: true
I20260808 07:11:15.059962 24338 leader_election.cc:288] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757), 6b4f319d65c6405ba0b0e7924e3609b5 (127.22.2.129:33775)
I20260808 07:11:15.059973 24101 raft_consensus.cc:2454] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 0.
I20260808 07:11:15.060214 23866 leader_election.cc:302] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4f319d65c6405ba0b0e7924e3609b5, 9caf0e680d284690bed4d39e7c38c098; no voters:
I20260808 07:11:15.060484 24335 raft_consensus.cc:2781] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:15.060611 24335 raft_consensus.cc:482] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:15.060721 24335 raft_consensus.cc:3037] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.061579 24335 raft_consensus.cc:504] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:15.061853 24101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c19987395e641f7a1514215087e8ea7" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caf0e680d284690bed4d39e7c38c098"
I20260808 07:11:15.061918 24258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c19987395e641f7a1514215087e8ea7" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e"
I20260808 07:11:15.061937 24101 raft_consensus.cc:3037] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.062005 24258 raft_consensus.cc:3037] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.062947 24258 raft_consensus.cc:2454] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 1.
I20260808 07:11:15.063004 24101 raft_consensus.cc:2454] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 1.
I20260808 07:11:15.063134 23864 leader_election.cc:302] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4f319d65c6405ba0b0e7924e3609b5, ce73d7c9195a4ce1b7b795f9eca6dd8e; no voters:
I20260808 07:11:15.063297 24344 raft_consensus.cc:2781] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:15.063308 24335 leader_election.cc:288] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 election: Requested vote from peers 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757), ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325)
I20260808 07:11:15.063465 24344 raft_consensus.cc:686] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 LEADER]: Becoming Leader. State: Replica: 6b4f319d65c6405ba0b0e7924e3609b5, State: Running, Role: LEADER
I20260808 07:11:15.063536 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.060116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33775 (local address 127.22.2.131:56181)
0808 07:11:15.060191 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:15.060201 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:15.060243 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:15.060440 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:15.060441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:15.060443 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:15.060515 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.060516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.061400 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.061401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.062499 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.062501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.063261 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.063262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.063347 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.063350 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:15.063408 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:11:15.063455 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 07:11:15.063589 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.060168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:56181 (local address 127.22.2.129:33775)
0808 07:11:15.060318 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.060320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.060327 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:15.060351 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.060351 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.060374 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.060425 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:15.060604 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.061310 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.062598 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.063149 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.063208 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.063217 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.063412 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.063452 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260808 07:11:15.063632 24344 consensus_queue.cc:237] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } }
I20260808 07:11:15.063832 23938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c19987395e641f7a1514215087e8ea7" candidate_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" is_pre_election: true
I20260808 07:11:15.064858 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.060180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35757 (local address 127.22.2.131:47859)
0808 07:11:15.060551 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:15.060562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:15.060625 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:15.060780 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:15.060780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:15.060783 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:15.060886 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.060888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.061737 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.061738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.063842 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.063845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.064644 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.064645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.064709 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.064712 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:15.064780 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:11:15.064808 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":93,"threads_started":1}
I20260808 07:11:15.064931 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.060522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:47859 (local address 127.22.2.130:35757)
0808 07:11:15.060567 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.060570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.060671 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:11:15.060693 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.060694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.060703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.060761 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:15.060964 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.061609 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.063992 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.064528 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.064613 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.064622 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.064791 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.064838 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:15.065227 24101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c19987395e641f7a1514215087e8ea7" candidate_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caf0e680d284690bed4d39e7c38c098" is_pre_election: true
I20260808 07:11:15.065325 24101 raft_consensus.cc:2379] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce73d7c9195a4ce1b7b795f9eca6dd8e in current term 1: Already voted for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in this term.
I20260808 07:11:15.065363 23760 catalog_manager.cc:5495] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b4f319d65c6405ba0b0e7924e3609b5 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } health_report { overall_health: HEALTHY } } }
I20260808 07:11:15.065543 24184 leader_election.cc:302] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce73d7c9195a4ce1b7b795f9eca6dd8e; no voters: 6b4f319d65c6405ba0b0e7924e3609b5, 9caf0e680d284690bed4d39e7c38c098
I20260808 07:11:15.065693 24338 raft_consensus.cc:2726] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 07:11:15.117871 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.115144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46070 (local address 127.22.2.129:33775)
0808 07:11:15.115225 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.115227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.115235 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:15.115279 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.115279 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.115297 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.115358 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:15.115680 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.116197 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.117037 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.117217 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.117279 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.117511 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:15.117608 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:15.117667 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.117789 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.117810 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:11:15.119848 24101 raft_consensus.cc:1261] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Refusing update from remote peer 6b4f319d65c6405ba0b0e7924e3609b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:15.119899 24258 raft_consensus.cc:1261] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Refusing update from remote peer 6b4f319d65c6405ba0b0e7924e3609b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:15.120195 24345 consensus_queue.cc:1035] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:15.120327 24344 consensus_queue.cc:1035] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:15.123936 24349 mvcc.cc:205] Tried to move back new op lower bound from 7316164915687960576 to 7316164915461918720. Current Snapshot: MvccSnapshot[applied={T|T < 7316164915687960576}]
I20260808 07:11:15.127163 23762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:15.127314 23762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:15.128324 23762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572: Sending DeleteTablet for 3 replicas of tablet 0c19987395e641f7a1514215087e8ea7
I20260808 07:11:15.128697 24078 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c19987395e641f7a1514215087e8ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:15 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:55322
I20260808 07:11:15.128720 23916 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c19987395e641f7a1514215087e8ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:15 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:46058
I20260808 07:11:15.128871 24235 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c19987395e641f7a1514215087e8ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:15 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:33664
I20260808 07:11:15.128995 24359 tablet_replica.cc:330] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: stopping tablet replica
I20260808 07:11:15.129164 24359 raft_consensus.cc:2229] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:15.129274 24358 tablet_replica.cc:330] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: stopping tablet replica
I20260808 07:11:15.129359 24359 raft_consensus.cc:2258] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:15.129535 23762 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:15.129750 23762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:15.129889 24359 ts_tablet_manager.cc:1905] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:15.129990 24358 raft_consensus.cc:2229] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:15.130115 24358 raft_consensus.cc:2258] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:15.131063 23916 tablet_service.cc:1467] Processing CreateTablet for tablet ba9583f9170646b79a8270f4351c9020 (DEFAULT_TABLE table=test-table [id=e6950fcfe5e64d9f80feb5abfbcdb9c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:15.131181 24235 tablet_service.cc:1467] Processing CreateTablet for tablet ba9583f9170646b79a8270f4351c9020 (DEFAULT_TABLE table=test-table [id=e6950fcfe5e64d9f80feb5abfbcdb9c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:15.131219 23916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba9583f9170646b79a8270f4351c9020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:15.131300 24235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba9583f9170646b79a8270f4351c9020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:15.131319 24078 tablet_service.cc:1467] Processing CreateTablet for tablet ba9583f9170646b79a8270f4351c9020 (DEFAULT_TABLE table=test-table [id=e6950fcfe5e64d9f80feb5abfbcdb9c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:15.131446 24078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba9583f9170646b79a8270f4351c9020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:15.132933 24332 tablet_bootstrap.cc:492] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: Bootstrap starting.
I20260808 07:11:15.133101 24331 tablet_bootstrap.cc:492] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Bootstrap starting.
I20260808 07:11:15.133498 24332 tablet_bootstrap.cc:654] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:15.133623 24331 tablet_bootstrap.cc:654] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:15.134071 24336 tablet_bootstrap.cc:492] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: Bootstrap starting.
I20260808 07:11:15.134234 24359 ts_tablet_manager.cc:1918] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:15.134327 24359 log.cc:1196] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/0c19987395e641f7a1514215087e8ea7
I20260808 07:11:15.134362 24332 tablet_bootstrap.cc:492] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: No bootstrap required, opened a new log
I20260808 07:11:15.134408 24332 ts_tablet_manager.cc:1397] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:15.134542 24336 tablet_bootstrap.cc:654] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:15.134552 24332 raft_consensus.cc:348] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.134615 24332 raft_consensus.cc:374] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:15.134636 24332 raft_consensus.cc:729] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4f319d65c6405ba0b0e7924e3609b5, State: Initialized, Role: FOLLOWER
I20260808 07:11:15.134681 24332 consensus_queue.cc:260] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.134871 24332 ts_tablet_manager.cc:1428] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:15.134665 24359 ts_tablet_manager.cc:1939] T 0c19987395e641f7a1514215087e8ea7 P 6b4f319d65c6405ba0b0e7924e3609b5: Deleting consensus metadata
I20260808 07:11:15.135592 23746 catalog_manager.cc:4841] TS 6b4f319d65c6405ba0b0e7924e3609b5 (127.22.2.129:33775): tablet 0c19987395e641f7a1514215087e8ea7 (table test-table [id=d5d91f98dd1e41609aa10bfe0bc5910d]) successfully deleted
I20260808 07:11:15.135828 24360 tablet_replica.cc:330] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: stopping tablet replica
I20260808 07:11:15.135917 24358 ts_tablet_manager.cc:1905] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:15.135967 24360 raft_consensus.cc:2229] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:15.136088 24360 raft_consensus.cc:2258] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:15.136154 24331 tablet_bootstrap.cc:492] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: No bootstrap required, opened a new log
I20260808 07:11:15.136216 24331 ts_tablet_manager.cc:1397] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260808 07:11:15.136389 24331 raft_consensus.cc:348] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.136482 24331 raft_consensus.cc:374] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:15.136510 24331 raft_consensus.cc:729] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce73d7c9195a4ce1b7b795f9eca6dd8e, State: Initialized, Role: FOLLOWER
I20260808 07:11:15.136590 24331 consensus_queue.cc:260] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.136710 24331 ts_tablet_manager.cc:1428] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.137502 24360 ts_tablet_manager.cc:1905] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:15.137876 24358 ts_tablet_manager.cc:1918] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:15.137940 24358 log.cc:1196] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/0c19987395e641f7a1514215087e8ea7
I20260808 07:11:15.138201 24358 ts_tablet_manager.cc:1939] T 0c19987395e641f7a1514215087e8ea7 P 9caf0e680d284690bed4d39e7c38c098: Deleting consensus metadata
I20260808 07:11:15.138610 24336 tablet_bootstrap.cc:492] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: No bootstrap required, opened a new log
I20260808 07:11:15.138659 24336 ts_tablet_manager.cc:1397] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 07:11:15.138718 23747 catalog_manager.cc:4841] TS 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757): tablet 0c19987395e641f7a1514215087e8ea7 (table test-table [id=d5d91f98dd1e41609aa10bfe0bc5910d]) successfully deleted
I20260808 07:11:15.138778 24336 raft_consensus.cc:348] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.138868 24336 raft_consensus.cc:374] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:15.138890 24336 raft_consensus.cc:729] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caf0e680d284690bed4d39e7c38c098, State: Initialized, Role: FOLLOWER
I20260808 07:11:15.138940 24336 consensus_queue.cc:260] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.139034 24336 ts_tablet_manager.cc:1428] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.139043 24360 ts_tablet_manager.cc:1918] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:15.139104 24360 log.cc:1196] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/0c19987395e641f7a1514215087e8ea7
I20260808 07:11:15.139374 24360 ts_tablet_manager.cc:1939] T 0c19987395e641f7a1514215087e8ea7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Deleting consensus metadata
I20260808 07:11:15.139951 23745 catalog_manager.cc:4841] TS ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325): tablet 0c19987395e641f7a1514215087e8ea7 (table test-table [id=d5d91f98dd1e41609aa10bfe0bc5910d]) successfully deleted
I20260808 07:11:15.223850 24344 raft_consensus.cc:482] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:15.223995 24344 raft_consensus.cc:504] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.224236 24344 leader_election.cc:288] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757), ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325)
I20260808 07:11:15.224459 24101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba9583f9170646b79a8270f4351c9020" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caf0e680d284690bed4d39e7c38c098" is_pre_election: true
I20260808 07:11:15.224565 24101 raft_consensus.cc:2454] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 0.
I20260808 07:11:15.224550 24258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba9583f9170646b79a8270f4351c9020" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" is_pre_election: true
I20260808 07:11:15.224682 24258 raft_consensus.cc:2454] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 0.
I20260808 07:11:15.224852 23866 leader_election.cc:302] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4f319d65c6405ba0b0e7924e3609b5, 9caf0e680d284690bed4d39e7c38c098; no voters:
I20260808 07:11:15.224948 24344 raft_consensus.cc:2781] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:15.224985 24344 raft_consensus.cc:482] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:15.225013 24344 raft_consensus.cc:3037] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.226038 24344 raft_consensus.cc:504] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.226189 24344 leader_election.cc:288] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 election: Requested vote from peers 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757), ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325)
I20260808 07:11:15.226364 24258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba9583f9170646b79a8270f4351c9020" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e"
I20260808 07:11:15.226385 24101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba9583f9170646b79a8270f4351c9020" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caf0e680d284690bed4d39e7c38c098"
I20260808 07:11:15.226469 24101 raft_consensus.cc:3037] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.226471 24258 raft_consensus.cc:3037] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.227140 24258 raft_consensus.cc:2454] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 1.
I20260808 07:11:15.227314 24101 raft_consensus.cc:2454] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 1.
I20260808 07:11:15.227320 23864 leader_election.cc:302] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4f319d65c6405ba0b0e7924e3609b5, ce73d7c9195a4ce1b7b795f9eca6dd8e; no voters:
I20260808 07:11:15.227433 24344 raft_consensus.cc:2781] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:15.227480 24344 raft_consensus.cc:686] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 LEADER]: Becoming Leader. State: Replica: 6b4f319d65c6405ba0b0e7924e3609b5, State: Running, Role: LEADER
I20260808 07:11:15.227537 24344 consensus_queue.cc:237] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.228236 23762 catalog_manager.cc:5495] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b4f319d65c6405ba0b0e7924e3609b5 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:11:15 dist-test-slave-6k22 krb5kdc[23712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:15.275089 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.268261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34327 (local address 127.0.0.1:45026)
0808 07:11:15.268465 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:15.268476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:15.268563 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:15.268731 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:15.268731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:15.269052 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:15.269100 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.269102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.269761 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.269762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.270281 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.270282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.270624 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.270625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.270676 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.271181 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:15.271185 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:15.272690 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:15.274107 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:15.274114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:15.274116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:15.274334 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:15.274513 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:15.274514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:15.274514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:15.274572 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:15.274800 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:15.274806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:15.274849 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:11:15.274900 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:11:15.274990 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":120,"threads_started":1}
I20260808 07:11:15.275159 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.268328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.22.2.190:34327)
0808 07:11:15.268442 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.268445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.268540 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:11:15.268636 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.268637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.268655 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.268711 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:15.269188 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.269674 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.270360 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.270540 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.270622 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.272788 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:15.272790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:15.272790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:15.272797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:15.274032 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.274397 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.274398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.274398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.274454 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.274628 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.274629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.274629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.274730 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:15.274796 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.274930 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.275037 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":57}
I20260808 07:11:15.277181 23762 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:45026:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:11:15.277420 23762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:15.278982 24078 tablet_service.cc:1467] Processing CreateTablet for tablet d6ae305382714eb681e1798aa6a023a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=88f7f2bc363d48baa0a4040c8a08af05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:15.278964 23916 tablet_service.cc:1467] Processing CreateTablet for tablet d6ae305382714eb681e1798aa6a023a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=88f7f2bc363d48baa0a4040c8a08af05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:15.279006 24235 tablet_service.cc:1467] Processing CreateTablet for tablet d6ae305382714eb681e1798aa6a023a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=88f7f2bc363d48baa0a4040c8a08af05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:15.279157 24235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ae305382714eb681e1798aa6a023a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:15.279158 23916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ae305382714eb681e1798aa6a023a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:15.279157 24078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ae305382714eb681e1798aa6a023a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:15.280537 24336 tablet_bootstrap.cc:492] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: Bootstrap starting.
I20260808 07:11:15.280762 24332 tablet_bootstrap.cc:492] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: Bootstrap starting.
I20260808 07:11:15.281004 24336 tablet_bootstrap.cc:654] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:15.281070 24332 tablet_bootstrap.cc:654] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:15.281409 24331 tablet_bootstrap.cc:492] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Bootstrap starting.
I20260808 07:11:15.281746 24332 tablet_bootstrap.cc:492] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: No bootstrap required, opened a new log
I20260808 07:11:15.281795 24332 ts_tablet_manager.cc:1397] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:15.281837 24331 tablet_bootstrap.cc:654] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:15.281924 24332 raft_consensus.cc:348] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.282029 24332 raft_consensus.cc:374] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:15.282070 24332 raft_consensus.cc:729] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4f319d65c6405ba0b0e7924e3609b5, State: Initialized, Role: FOLLOWER
I20260808 07:11:15.282130 24332 consensus_queue.cc:260] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.282259 24332 ts_tablet_manager.cc:1428] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.282311 24344 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.282413 24344 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:15.282466 24336 tablet_bootstrap.cc:492] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: No bootstrap required, opened a new log
I20260808 07:11:15.282522 24336 ts_tablet_manager.cc:1397] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:15.282681 24336 raft_consensus.cc:348] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.282760 24336 raft_consensus.cc:374] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:15.282812 24336 raft_consensus.cc:729] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caf0e680d284690bed4d39e7c38c098, State: Initialized, Role: FOLLOWER
I20260808 07:11:15.282858 24336 consensus_queue.cc:260] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.282896 24331 tablet_bootstrap.cc:492] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: No bootstrap required, opened a new log
I20260808 07:11:15.282943 24331 ts_tablet_manager.cc:1397] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 07:11:15.282959 24336 ts_tablet_manager.cc:1428] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.283023 24340 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.283114 24340 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:15.283102 24331 raft_consensus.cc:348] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.283175 24331 raft_consensus.cc:374] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:15.283213 24331 raft_consensus.cc:729] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce73d7c9195a4ce1b7b795f9eca6dd8e, State: Initialized, Role: FOLLOWER
I20260808 07:11:15.283278 24331 consensus_queue.cc:260] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.283386 24331 ts_tablet_manager.cc:1428] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.283438 24338 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.283533 24338 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:15.324746 24344 raft_consensus.cc:482] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:15.324856 24344 raft_consensus.cc:504] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.325014 24344 leader_election.cc:288] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757), ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325)
I20260808 07:11:15.325335 24101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ae305382714eb681e1798aa6a023a7" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caf0e680d284690bed4d39e7c38c098" is_pre_election: true
I20260808 07:11:15.325335 24258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ae305382714eb681e1798aa6a023a7" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" is_pre_election: true
I20260808 07:11:15.325464 24101 raft_consensus.cc:2454] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 0.
I20260808 07:11:15.325464 24258 raft_consensus.cc:2454] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 0.
I20260808 07:11:15.325722 23864 leader_election.cc:302] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4f319d65c6405ba0b0e7924e3609b5, ce73d7c9195a4ce1b7b795f9eca6dd8e; no voters:
I20260808 07:11:15.325852 24344 raft_consensus.cc:2781] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:15.325901 24344 raft_consensus.cc:482] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:15.325930 24344 raft_consensus.cc:3037] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.326709 24344 raft_consensus.cc:504] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.326881 24344 leader_election.cc:288] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 election: Requested vote from peers 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757), ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325)
I20260808 07:11:15.327059 24258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ae305382714eb681e1798aa6a023a7" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e"
I20260808 07:11:15.327147 24258 raft_consensus.cc:3037] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.327096 24101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ae305382714eb681e1798aa6a023a7" candidate_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caf0e680d284690bed4d39e7c38c098"
I20260808 07:11:15.327196 24101 raft_consensus.cc:3037] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.327880 24258 raft_consensus.cc:2454] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 1.
I20260808 07:11:15.327922 24101 raft_consensus.cc:2454] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4f319d65c6405ba0b0e7924e3609b5 in term 1.
I20260808 07:11:15.328064 23864 leader_election.cc:302] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4f319d65c6405ba0b0e7924e3609b5, ce73d7c9195a4ce1b7b795f9eca6dd8e; no voters:
I20260808 07:11:15.328172 24344 raft_consensus.cc:2781] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:15.328265 24344 raft_consensus.cc:686] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 LEADER]: Becoming Leader. State: Replica: 6b4f319d65c6405ba0b0e7924e3609b5, State: Running, Role: LEADER
I20260808 07:11:15.328333 24344 consensus_queue.cc:237] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } }
I20260808 07:11:15.328701 24345 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: TxnStatusTablet state changed. Reason: New leader 6b4f319d65c6405ba0b0e7924e3609b5. Latest consensus state: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.328804 24345 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:15.329056 23760 catalog_manager.cc:5495] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b4f319d65c6405ba0b0e7924e3609b5 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:15.329401 24377 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:11:15.329552 24377 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:11:15.343886 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.341431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33775 (local address 127.0.0.1:46076)
0808 07:11:15.341533 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:15.341547 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:15.341597 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:15.341741 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:15.341742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:15.341743 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:15.341799 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.341802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.342434 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.342436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.343025 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.343026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.343389 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.343390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.343449 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.343469 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:15.343701 (+ 232us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:15.343712 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:11:15.343762 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:11:15.343815 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:11:15.343878 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.341499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46076 (local address 127.22.2.129:33775)
0808 07:11:15.341564 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.341566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.341577 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:15.341662 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.341663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.341677 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.341734 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:15.341866 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.342350 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.343114 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.343316 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.343390 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.343589 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:15.343631 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:15.343699 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.343767 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.343789 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:11:15.345027 24258 raft_consensus.cc:1261] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Refusing update from remote peer 6b4f319d65c6405ba0b0e7924e3609b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:15.345048 24101 raft_consensus.cc:1261] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Refusing update from remote peer 6b4f319d65c6405ba0b0e7924e3609b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:15.345263 24345 consensus_queue.cc:1035] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:15.345361 24344 consensus_queue.cc:1035] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:15.346022 24338 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: TxnStatusTablet state changed. Reason: New leader 6b4f319d65c6405ba0b0e7924e3609b5. Latest consensus state: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.346125 24338 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:15.346316 24340 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: TxnStatusTablet state changed. Reason: New leader 6b4f319d65c6405ba0b0e7924e3609b5. Latest consensus state: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.346406 24340 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:15.346844 24344 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.346925 24344 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:15.346961 24335 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.347023 24335 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P 6b4f319d65c6405ba0b0e7924e3609b5: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:15.346987 24348 mvcc.cc:205] Tried to move back new op lower bound from 7316164916611379200 to 7316164916545310720. Current Snapshot: MvccSnapshot[applied={T|T < 7316164916611379200 or (T in {7316164916611379200})}]
I20260808 07:11:15.347105 24350 mvcc.cc:205] Tried to move back new op lower bound from 7316164916611379200 to 7316164916545310720. Current Snapshot: MvccSnapshot[applied={T|T < 7316164916611379200}]
I20260808 07:11:15.347234 24338 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.347302 24338 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P ce73d7c9195a4ce1b7b795f9eca6dd8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:15.347832 24340 tablet_replica.cc:439] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4f319d65c6405ba0b0e7924e3609b5" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33775 } } peers { permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 } } peers { permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 } } }
I20260808 07:11:15.347919 24340 tablet_replica.cc:442] T d6ae305382714eb681e1798aa6a023a7 P 9caf0e680d284690bed4d39e7c38c098: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:15.353258 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.350821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:57229 (local address 127.22.2.129:33775)
0808 07:11:15.350859 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.350861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.350990 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:11:15.351054 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.351055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.351065 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.351116 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:15.351217 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.351751 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.352531 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.352771 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.352825 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.352999 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:15.353030 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:15.353072 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.353152 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.353168 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:11:15.353475 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.350754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33775 (local address 127.22.2.129:57229)
0808 07:11:15.350917 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:15.350929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:15.351007 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:15.351123 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:15.351124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:15.351126 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:15.351169 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.351170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.351824 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.351825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.352435 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.352436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.352837 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.352838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.352886 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.352909 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:15.353084 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:15.353085 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:11:15.353127 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 07:11:15.353175 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
I20260808 07:11:15.355717 24258 raft_consensus.cc:1261] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Refusing update from remote peer 6b4f319d65c6405ba0b0e7924e3609b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:15.355823 24101 raft_consensus.cc:1261] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Refusing update from remote peer 6b4f319d65c6405ba0b0e7924e3609b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:15.355923 24335 consensus_queue.cc:1035] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce73d7c9195a4ce1b7b795f9eca6dd8e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:15.356050 24335 consensus_queue.cc:1035] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caf0e680d284690bed4d39e7c38c098" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:15.367383 23760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:15.367506 23760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:15.368217 23760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 613744e19d484e31a2baf77e1c854572: Sending DeleteTablet for 3 replicas of tablet ba9583f9170646b79a8270f4351c9020
I20260808 07:11:15.368539 24235 tablet_service.cc:1514] Processing DeleteTablet for tablet ba9583f9170646b79a8270f4351c9020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:15 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:33664
I20260808 07:11:15.368623 24078 tablet_service.cc:1514] Processing DeleteTablet for tablet ba9583f9170646b79a8270f4351c9020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:15 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:55322
I20260808 07:11:15.368728 24360 tablet_replica.cc:330] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: stopping tablet replica
I20260808 07:11:15.368789 24360 raft_consensus.cc:2229] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:15.368824 24360 raft_consensus.cc:2258] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:15.368898 24358 tablet_replica.cc:330] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: stopping tablet replica
I20260808 07:11:15.368950 24358 raft_consensus.cc:2229] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:15.368988 24358 raft_consensus.cc:2258] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:15.369129 23916 tablet_service.cc:1514] Processing DeleteTablet for tablet ba9583f9170646b79a8270f4351c9020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:15 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:46058
I20260808 07:11:15.369195 24360 ts_tablet_manager.cc:1905] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:15.369473 24358 ts_tablet_manager.cc:1905] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:15.370935 24360 ts_tablet_manager.cc:1918] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:15.371021 24360 log.cc:1196] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/ba9583f9170646b79a8270f4351c9020
I20260808 07:11:15.371127 24358 ts_tablet_manager.cc:1918] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:15.371198 24358 log.cc:1196] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/ba9583f9170646b79a8270f4351c9020
I20260808 07:11:15.371270 24360 ts_tablet_manager.cc:1939] T ba9583f9170646b79a8270f4351c9020 P ce73d7c9195a4ce1b7b795f9eca6dd8e: Deleting consensus metadata
I20260808 07:11:15.371409 24358 ts_tablet_manager.cc:1939] T ba9583f9170646b79a8270f4351c9020 P 9caf0e680d284690bed4d39e7c38c098: Deleting consensus metadata
I20260808 07:11:15.371541 24359 tablet_replica.cc:330] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: stopping tablet replica
I20260808 07:11:15.371621 23745 catalog_manager.cc:4841] TS ce73d7c9195a4ce1b7b795f9eca6dd8e (127.22.2.131:37325): tablet ba9583f9170646b79a8270f4351c9020 (table test-table [id=e6950fcfe5e64d9f80feb5abfbcdb9c2]) successfully deleted
I20260808 07:11:15.371649 24359 raft_consensus.cc:2229] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:15.371722 24359 raft_consensus.cc:2258] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:15.371935 23747 catalog_manager.cc:4841] TS 9caf0e680d284690bed4d39e7c38c098 (127.22.2.130:35757): tablet ba9583f9170646b79a8270f4351c9020 (table test-table [id=e6950fcfe5e64d9f80feb5abfbcdb9c2]) successfully deleted
I20260808 07:11:15.372074 24359 ts_tablet_manager.cc:1905] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:15.373262 24359 ts_tablet_manager.cc:1918] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:15.373311 24359 log.cc:1196] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/ba9583f9170646b79a8270f4351c9020
I20260808 07:11:15.373531 24359 ts_tablet_manager.cc:1939] T ba9583f9170646b79a8270f4351c9020 P 6b4f319d65c6405ba0b0e7924e3609b5: Deleting consensus metadata
I20260808 07:11:15.373862 23746 catalog_manager.cc:4841] TS 6b4f319d65c6405ba0b0e7924e3609b5 (127.22.2.129:33775): tablet ba9583f9170646b79a8270f4351c9020 (table test-table [id=e6950fcfe5e64d9f80feb5abfbcdb9c2]) successfully deleted
Aug 08 07:11:15 dist-test-slave-6k22 krb5kdc[23712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:15.385135 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.369316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.22.2.129:33775)
0808 07:11:15.369366 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.369369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.375777 (+ 6408us) server_negotiation.cc:407] Connection header received
0808 07:11:15.375840 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.375841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.375852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.375906 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:15.376595 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.377101 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.378151 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.378369 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.378431 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.382053 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:15.382066 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:15.382066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:15.382074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:15.383917 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.384332 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.384335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.384335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.384392 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.384592 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.384593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.384593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.384776 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:15.384886 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.384998 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.385063 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:11:15.385502 23984 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46078
I20260808 07:11:15.393539 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.386590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.22.2.190:34327)
0808 07:11:15.386657 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.386659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.386669 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:15.386720 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.386721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.386732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.386796 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:15.387292 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.388027 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.388704 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.389012 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.389086 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.390360 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:15.390362 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:15.390362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:15.390370 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:15.391927 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.392506 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.392507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.392508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.392613 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.392897 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.392898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.392898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.393074 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:15.393204 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.393308 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.393425 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 07:11:15.393841 23760 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45028
I20260808 07:11:15.394742 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 23842
I20260808 07:11:15.409917 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 23999
I20260808 07:11:15.417007 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 24162
I20260808 07:11:15.423306 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 23726
2026-08-08T07:11:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2538 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:15 dist-test-slave-6k22 krb5kdc[24398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:15 dist-test-slave-6k22 krb5kdc[24398](info): set up 2 sockets
Aug 08 07:11:15 dist-test-slave-6k22 krb5kdc[24398](info): commencing operation
krb5kdc: starting...
W20260808 07:11:16.542171 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:16 dist-test-slave-6k22 krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:16.666569 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37667
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:37667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:16.749425 24415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:16.749682 24415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:16.749730 24415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:16.750874 24415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:16.750945 24415 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:11:16.750968 24415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:16.751000 24415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:16.751044 24415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:16.751075 24415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:16.752672 24415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:37667
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:16.752952 24415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:16.753180 24415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:16.758095 24422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.758093 24424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.758098 24421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.758378 24415 server_base.cc:1034] running on GCE node
I20260808 07:11:16.758773 24415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:16.759073 24415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:16.760252 24415 hybrid_clock.cc:648] HybridClock initialized: now 1786173076760225 us; error 39 us; skew 500 ppm
Aug 08 07:11:16 dist-test-slave-6k22 krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:16.762792 24415 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:16.763276 24415 webserver.cc:460] Webserver started at http://127.22.2.190:43015/ using document root <none> and password file <none>
I20260808 07:11:16.763514 24415 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:16.763609 24415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:16.763774 24415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:16.764859 24415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "c556c1164e324d0d9b778c6e748bb125"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-6k22"
server_key: "8ea1bf2af46a41616d117e3cdb0a3fd8"
server_key_iv: "fb4dda10d5025946620d13bc558cdbc7"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.765239 24415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "c556c1164e324d0d9b778c6e748bb125"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-6k22"
server_key: "8ea1bf2af46a41616d117e3cdb0a3fd8"
server_key_iv: "fb4dda10d5025946620d13bc558cdbc7"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.766698 24415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:16.767491 24431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.767722 24415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.767838 24415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "c556c1164e324d0d9b778c6e748bb125"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-6k22"
server_key: "8ea1bf2af46a41616d117e3cdb0a3fd8"
server_key_iv: "fb4dda10d5025946620d13bc558cdbc7"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.767918 24415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:16.782361 24415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:16.784260 24415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:16.784425 24415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:16.789932 24415 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:37667
I20260808 07:11:16.789990 24504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:37667 every 8 connection(s)
I20260808 07:11:16.790354 24415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:16.791026 24505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:16.792032 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 24415
I20260808 07:11:16.792152 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:16.792366 22538 external_mini_cluster.cc:1428] Setting key a48b9500de406b4b473b5416f12015f2
I20260808 07:11:16.793659 24505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: Bootstrap starting.
I20260808 07:11:16.794456 24505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:16.794788 24505 log.cc:824] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:16.795511 24505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: No bootstrap required, opened a new log
I20260808 07:11:16.797935 24505 raft_consensus.cc:348] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c556c1164e324d0d9b778c6e748bb125" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37667 } }
I20260808 07:11:16.798148 24505 raft_consensus.cc:374] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:16.798193 24505 raft_consensus.cc:729] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c556c1164e324d0d9b778c6e748bb125, State: Initialized, Role: FOLLOWER
I20260808 07:11:16.798354 24505 consensus_queue.cc:260] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c556c1164e324d0d9b778c6e748bb125" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37667 } }
I20260808 07:11:16.798450 24505 raft_consensus.cc:388] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:16.798486 24505 raft_consensus.cc:482] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:16.798540 24505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:16.799311 24505 raft_consensus.cc:504] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c556c1164e324d0d9b778c6e748bb125" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37667 } }
I20260808 07:11:16.799399 24505 leader_election.cc:302] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c556c1164e324d0d9b778c6e748bb125; no voters:
I20260808 07:11:16.799582 24505 leader_election.cc:288] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:16.799616 24513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:16.799733 24513 raft_consensus.cc:686] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [term 1 LEADER]: Becoming Leader. State: Replica: c556c1164e324d0d9b778c6e748bb125, State: Running, Role: LEADER
I20260808 07:11:16.799831 24513 consensus_queue.cc:237] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c556c1164e324d0d9b778c6e748bb125" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37667 } }
Aug 08 07:11:16 dist-test-slave-6k22 krb5kdc[24398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:16.800045 24505 sys_catalog.cc:564] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:16.800130 24513 sys_catalog.cc:455] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c556c1164e324d0d9b778c6e748bb125. Latest consensus state: current_term: 1 leader_uuid: "c556c1164e324d0d9b778c6e748bb125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c556c1164e324d0d9b778c6e748bb125" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37667 } } }
I20260808 07:11:16.800171 24515 sys_catalog.cc:455] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c556c1164e324d0d9b778c6e748bb125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c556c1164e324d0d9b778c6e748bb125" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37667 } } }
I20260808 07:11:16.800199 24513 sys_catalog.cc:458] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:16.800406 24515 sys_catalog.cc:458] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:16.800673 24517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:16.801991 24517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:16.804958 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.792947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52348 (local address 127.22.2.190:37667)
0808 07:11:16.793129 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.793137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.793168 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:11:16.793603 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.793609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.793656 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.793879 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.794691 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.795849 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.796995 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.797460 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.797546 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.800714 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:16.800718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:16.800718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:16.800731 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:16.802924 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.803407 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.803411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.803411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.803536 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.803836 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.803837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.803838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.804087 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:16.804264 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:16.804554 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 07:11:16.804713 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260808 07:11:16.805261 24517 catalog_manager.cc:1347] Generated new cluster ID: 5d566488dc3a416ba482be6a18735fce
I20260808 07:11:16.805310 24517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:16.823912 24517 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:16.824536 24517 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:16.827226 24517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: Generated new TSK 0
I20260808 07:11:16.827421 24517 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:16.861853 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37667
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:16.945153 24534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:16.945367 24534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:16.945393 24534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:16.946568 24534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:16.946655 24534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:16.946702 24534 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:11:16.946733 24534 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:16.946789 24534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:16.948285 24534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:16.948527 24534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:16.948791 24534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:16.954259 24539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.954259 24540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.954331 24542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.954728 24534 server_base.cc:1034] running on GCE node
I20260808 07:11:16.955062 24534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:16.955341 24534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:16.956534 24534 hybrid_clock.cc:648] HybridClock initialized: now 1786173076956507 us; error 54 us; skew 500 ppm
Aug 08 07:11:16 dist-test-slave-6k22 krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:16.959193 24534 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:16.959683 24534 webserver.cc:460] Webserver started at http://127.22.2.129:40911/ using document root <none> and password file <none>
I20260808 07:11:16.959873 24534 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:16.959914 24534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:16.960006 24534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:16.961040 24534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "5656022f91214a69b5c23b98bbe31009"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-6k22"
server_key: "621215a2a94f384f504445e6bf4bc20f"
server_key_iv: "85a7a39235a4cc0d34aa43f5cca594aa"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.961349 24534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "5656022f91214a69b5c23b98bbe31009"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-6k22"
server_key: "621215a2a94f384f504445e6bf4bc20f"
server_key_iv: "85a7a39235a4cc0d34aa43f5cca594aa"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.962752 24534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:16.963624 24550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.963802 24534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:16.963893 24534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "5656022f91214a69b5c23b98bbe31009"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-6k22"
server_key: "621215a2a94f384f504445e6bf4bc20f"
server_key_iv: "85a7a39235a4cc0d34aa43f5cca594aa"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.963950 24534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:16.976200 24534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:16.978520 24534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:16.978637 24534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:16.979115 24534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:16.979151 24534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.979174 24534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:16.979233 24534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.986064 24534 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:37231
I20260808 07:11:16.986150 24680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:37231 every 8 connection(s)
I20260808 07:11:16.986444 24534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:11:16 dist-test-slave-6k22 krb5kdc[24398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:16.987326 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 24534
I20260808 07:11:16.987437 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:16.987671 22538 external_mini_cluster.cc:1428] Setting key 48383f88836512657a6e6fcc9561e825
I20260808 07:11:16.990286 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.979691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:35329 (local address 127.22.2.190:37667)
0808 07:11:16.979766 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.979768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.980104 (+ 336us) server_negotiation.cc:407] Connection header received
0808 07:11:16.980659 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.980660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.980675 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.980765 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.981635 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.982497 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.984143 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.984495 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.984572 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.987316 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:16.987318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:16.987318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:16.987328 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:16.988852 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.989296 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.989298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.989298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.989360 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.989616 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.989617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.989618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.989735 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:16.989814 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:16.990103 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 07:11:16.990195 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:11:16.990474 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.979788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37667 (local address 127.22.2.129:35329)
0808 07:11:16.979964 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.980006 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.980542 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.980861 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.980866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.981233 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.981523 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.981531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.982604 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.982606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.984013 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.984016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.984595 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.984596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.984814 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.985442 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.985453 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.987211 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:16.988960 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:16.988968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:16.988970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:16.989197 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:16.989430 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:16.989431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:16.989431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:16.989548 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:16.989817 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:16.989825 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:16.989967 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 07:11:16.990069 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:11:16.990231 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
I20260808 07:11:16.991052 24681 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37667
I20260808 07:11:16.991161 24681 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:16.991348 24681 heartbeater.cc:511] Master 127.22.2.190:37667 requested a full tablet report, sending...
I20260808 07:11:16.991916 24450 ts_manager.cc:194] Registered new tserver with Master: 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231)
I20260808 07:11:16.992836 24450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:35329
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 07:11:16.998477 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.992240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:47237 (local address 127.22.2.190:37667)
0808 07:11:16.992302 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.992304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.992535 (+ 231us) server_negotiation.cc:407] Connection header received
0808 07:11:16.992566 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.992567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.992580 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.992651 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.993037 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.993825 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.994567 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.994883 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.994959 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.995955 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:16.995956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:16.995957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:16.995964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:16.997504 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.997826 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.997827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.997827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.997876 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.998011 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.998011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.998012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.998101 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:16.998165 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:16.998358 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 07:11:16.998418 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:11:16.998584 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.992293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37667 (local address 127.22.2.129:47237)
0808 07:11:16.992453 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.992464 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.992525 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.992655 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.992656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.992863 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.992926 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.992929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.993962 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.993963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.994474 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.994476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.994989 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.994990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.995058 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.995536 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.995541 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.995872 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:16.997579 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:16.997580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:16.997580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:16.997762 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:16.997921 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:16.997921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:16.997921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:16.997964 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:16.998160 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:16.998161 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:16.998219 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 07:11:16.998314 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 07:11:16.998458 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:17.018491 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37667
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:17.102967 24690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:17.103170 24690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:17.103195 24690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:17.104292 24690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:17.104329 24690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:17.104350 24690 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:11:17.104362 24690 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:17.104379 24690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:17.105764 24690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:17.105952 24690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:17.106132 24690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:17.111258 24697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:17.111203 24699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:17.111212 24696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:17.111389 24690 server_base.cc:1034] running on GCE node
I20260808 07:11:17.111711 24690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:17.111975 24690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:17.113137 24690 hybrid_clock.cc:648] HybridClock initialized: now 1786173077113117 us; error 38 us; skew 500 ppm
Aug 08 07:11:17 dist-test-slave-6k22 krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173077, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:17.115630 24690 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:17.116089 24690 webserver.cc:460] Webserver started at http://127.22.2.130:40385/ using document root <none> and password file <none>
I20260808 07:11:17.116339 24690 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:17.116385 24690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:17.116533 24690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:17.117581 24690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "e1c3af9dc427446fa78ebdef636c9052"
format_stamp: "Formatted at 2026-08-08 07:11:17 on dist-test-slave-6k22"
server_key: "813f894d27c5ceea66757384a4b419f6"
server_key_iv: "75ff67e51ea61f0e5ad55b170e0c3457"
server_key_version: "encryptionkey@0"
I20260808 07:11:17.117934 24690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "e1c3af9dc427446fa78ebdef636c9052"
format_stamp: "Formatted at 2026-08-08 07:11:17 on dist-test-slave-6k22"
server_key: "813f894d27c5ceea66757384a4b419f6"
server_key_iv: "75ff67e51ea61f0e5ad55b170e0c3457"
server_key_version: "encryptionkey@0"
I20260808 07:11:17.119284 24690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:17.120198 24708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:17.120400 24690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:11:17.120473 24690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "e1c3af9dc427446fa78ebdef636c9052"
format_stamp: "Formatted at 2026-08-08 07:11:17 on dist-test-slave-6k22"
server_key: "813f894d27c5ceea66757384a4b419f6"
server_key_iv: "75ff67e51ea61f0e5ad55b170e0c3457"
server_key_version: "encryptionkey@0"
I20260808 07:11:17.120564 24690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:17.134018 24690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:17.140384 24690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:17.140571 24690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:17.141186 24690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:17.141229 24690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:17.141253 24690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:17.141287 24690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:11:17 dist-test-slave-6k22 krb5kdc[24398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173077, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:17.148789 24690 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:44125
I20260808 07:11:17.149174 24690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:17.149166 24835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:44125 every 8 connection(s)
I20260808 07:11:17.151614 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.141748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53877 (local address 127.22.2.190:37667)
0808 07:11:17.141799 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.141801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.142047 (+ 246us) server_negotiation.cc:407] Connection header received
0808 07:11:17.142594 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.142595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.142609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.142675 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.143589 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.144478 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.145511 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.145765 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.145866 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.148576 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:17.148578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:17.148579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:17.148587 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:17.150248 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.150741 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.150742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.150742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.150801 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.151019 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.151020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.151020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.151148 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:17.151199 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.151436 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.151498 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:17.151825 24718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.141744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37667 (local address 127.22.2.130:53877)
0808 07:11:17.141929 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.141965 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.142488 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.142747 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.142752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.143144 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:17.143453 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.143460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.144598 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.144599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.145416 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.145418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.145908 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.145910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.146091 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.146692 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:17.146703 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:17.148464 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:17.150351 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.150357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.150359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.150651 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.150863 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.150864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.150864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.150964 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.151202 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:17.151207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:17.151325 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.151407 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.151530 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
I20260808 07:11:17.152366 24836 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37667
I20260808 07:11:17.152451 24836 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:17.152714 24836 heartbeater.cc:511] Master 127.22.2.190:37667 requested a full tablet report, sending...
I20260808 07:11:17.153105 24451 ts_manager.cc:194] Registered new tserver with Master: e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125)
I20260808 07:11:17.153600 24451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:53877
I20260808 07:11:17.154218 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 24690
I20260808 07:11:17.154333 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:17.154526 22538 external_mini_cluster.cc:1428] Setting key ab15a3670defe4c04c5f59ae8e9e33dc
I20260808 07:11:17.159394 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.153407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53285 (local address 127.22.2.190:37667)
0808 07:11:17.153444 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.153446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.153575 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:11:17.153657 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.153658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.153667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.153714 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.154132 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.154838 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.155590 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.155889 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.155964 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.156833 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:17.156835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:17.156835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:17.156842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:17.158226 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.158578 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.158579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.158579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.158634 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.158843 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.158844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.158844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.158991 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:17.159069 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.159227 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.159304 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:11:17.159409 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.153351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37667 (local address 127.22.2.130:53285)
0808 07:11:17.153513 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.153522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.153590 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.153772 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.153773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.153974 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:17.154045 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.154047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.154922 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.154923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.155484 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.155485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.155987 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.155988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.156055 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.156464 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:17.156468 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:17.156746 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:17.158324 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.158325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.158325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.158504 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.158710 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.158711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.158711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.158770 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.159062 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:17.159063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:17.159110 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.159199 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.159315 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:17.181890 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37667
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:17.265322 24846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:17.265527 24846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:17.265553 24846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:17.266772 24846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:17.266814 24846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:17.266834 24846 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:11:17.266876 24846 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:17.266920 24846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:17.268416 24846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:17.268672 24846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:17.268910 24846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:17.274123 24846 server_base.cc:1034] running on GCE node
W20260808 07:11:17.274082 24851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:17.274089 24854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:17.274106 24852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:17.274485 24846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:17.274778 24846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:17.275949 24846 hybrid_clock.cc:648] HybridClock initialized: now 1786173077275922 us; error 55 us; skew 500 ppm
Aug 08 07:11:17 dist-test-slave-6k22 krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173077, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:17.278473 24846 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:17.278942 24846 webserver.cc:460] Webserver started at http://127.22.2.131:45839/ using document root <none> and password file <none>
I20260808 07:11:17.279209 24846 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:17.279281 24846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:17.279433 24846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:17.280568 24846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "915828ad95484fa99a950d110fdc8f16"
format_stamp: "Formatted at 2026-08-08 07:11:17 on dist-test-slave-6k22"
server_key: "92414442ffd2cb91cc6f841df07263c5"
server_key_iv: "4e955b24c834f1e828a9506c6b359550"
server_key_version: "encryptionkey@0"
I20260808 07:11:17.280947 24846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "915828ad95484fa99a950d110fdc8f16"
format_stamp: "Formatted at 2026-08-08 07:11:17 on dist-test-slave-6k22"
server_key: "92414442ffd2cb91cc6f841df07263c5"
server_key_iv: "4e955b24c834f1e828a9506c6b359550"
server_key_version: "encryptionkey@0"
I20260808 07:11:17.282285 24846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:17.283030 24862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:17.283202 24846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:17.283296 24846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "915828ad95484fa99a950d110fdc8f16"
format_stamp: "Formatted at 2026-08-08 07:11:17 on dist-test-slave-6k22"
server_key: "92414442ffd2cb91cc6f841df07263c5"
server_key_iv: "4e955b24c834f1e828a9506c6b359550"
server_key_version: "encryptionkey@0"
I20260808 07:11:17.283380 24846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:17.301409 24846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:17.303661 24846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:17.303829 24846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:17.304369 24846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:17.304435 24846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:17.304482 24846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:17.304510 24846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:11:17 dist-test-slave-6k22 krb5kdc[24398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173077, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:17.312057 24846 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:38571
I20260808 07:11:17.312359 24988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:38571 every 8 connection(s)
I20260808 07:11:17.312525 24846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:17.314502 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.304895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:59673 (local address 127.22.2.190:37667)
0808 07:11:17.304942 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.304944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.305209 (+ 265us) server_negotiation.cc:407] Connection header received
0808 07:11:17.305739 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.305740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.305754 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.305802 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.306665 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.307503 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.308592 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.308946 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.309033 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.311734 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:17.311736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:17.311736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:17.311753 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:17.313384 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.313757 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.313758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.313758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.313810 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.313983 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.313983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.313984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.314084 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:17.314133 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.314375 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.314437 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:11:17.314754 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.304905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37667 (local address 127.22.2.131:59673)
0808 07:11:17.305092 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.305132 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.305636 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.305873 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.305876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.306245 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:17.306541 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.306548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.307648 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.307650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.308518 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.308520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.309046 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.309047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.309185 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.309735 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:17.309747 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:17.311620 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:17.313459 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.313463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.313465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.313686 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.313851 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.313851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.313852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.313935 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.314142 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:17.314148 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:17.314264 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.314350 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.314471 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":114,"threads_started":1}
I20260808 07:11:17.315239 24989 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37667
I20260808 07:11:17.315326 24989 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:17.315524 24989 heartbeater.cc:511] Master 127.22.2.190:37667 requested a full tablet report, sending...
I20260808 07:11:17.315950 24450 ts_manager.cc:194] Registered new tserver with Master: 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571)
I20260808 07:11:17.316461 24450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:59673
I20260808 07:11:17.317327 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 24846
I20260808 07:11:17.317409 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:17.317615 22538 external_mini_cluster.cc:1428] Setting key b86b6e68d5f8e1bbe645ae37da5849ef
I20260808 07:11:17.318471 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:11:17.322640 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.316411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:57145 (local address 127.22.2.190:37667)
0808 07:11:17.316450 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.316452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.316560 (+ 108us) server_negotiation.cc:407] Connection header received
0808 07:11:17.316770 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.316770 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.316779 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.316848 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.317240 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.317864 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.318754 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.319054 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.319124 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.320101 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:17.320102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:17.320103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:17.320110 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:17.321538 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.321940 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.321941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.321942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.321994 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.322207 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.322208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.322208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.322306 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:17.322360 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.322524 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.322581 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:17.322759 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.316355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37667 (local address 127.22.2.131:57145)
0808 07:11:17.316496 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.316506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.316587 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.316871 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.316872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.317056 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:17.317113 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.317150 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.317944 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.317945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.318684 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.318686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.319138 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.319139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.319193 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.319686 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:17.319691 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:17.320008 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:17.321622 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.321623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.321624 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.321855 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.322063 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.322064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.322064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.322143 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.322370 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:17.322371 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:17.322423 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.322507 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.322624 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":90,"threads_started":1}
Aug 08 07:11:17 dist-test-slave-6k22 krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:11:17 dist-test-slave-6k22 krb5kdc[24398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:17.343835 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.336881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52356 (local address 127.22.2.190:37667)
0808 07:11:17.336949 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.336951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.336960 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:17.336985 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.336985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.336994 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.337044 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.337628 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.338147 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.338874 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.339058 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.339114 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.341395 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:17.341396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:17.341397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:17.341404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:17.342714 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.343130 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.343130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.343131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.343177 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.343356 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.343357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.343357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.343454 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:17.343531 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.343707 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.343767 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:11:17.345847 24450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:17.347450 24450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:17.354346 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.350145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37231 (local address 127.0.0.1:52988)
0808 07:11:17.350310 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.350352 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.350416 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.350951 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.350954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.350956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:17.351045 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.351047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.352190 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.352191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.353357 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.353361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.354021 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.354022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.354109 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.354122 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.354209 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.354266 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260808 07:11:17.354684 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.350527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52988 (local address 127.22.2.129:37231)
0808 07:11:17.350780 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.350785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.350797 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:11:17.350843 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.350845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.350862 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.350936 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:17.351184 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.352101 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.353428 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.353936 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.354023 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.354086 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.354550 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.354592 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":131,"threads_started":1}
I20260808 07:11:17.355506 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.350487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.22.2.130:44125)
0808 07:11:17.350811 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.350816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.351179 (+ 363us) server_negotiation.cc:407] Connection header received
0808 07:11:17.351228 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.351231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.351248 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.351327 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:17.351494 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.352750 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.354208 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.354983 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.355108 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.355173 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.355368 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.355404 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":92,"threads_started":1}
I20260808 07:11:17.355512 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.350372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44125 (local address 127.0.0.1:50690)
0808 07:11:17.351025 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.351035 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.351085 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.351345 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.351346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.351348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:17.351407 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.351409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.352833 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.352833 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.354111 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.354114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.355208 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.355209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.355290 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.355296 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.355350 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.355396 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":56,"thread_start_us":73,"threads_started":1}
I20260808 07:11:17.355526 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.350150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38571 (local address 127.0.0.1:60370)
0808 07:11:17.350493 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.350505 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.350560 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.351437 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.351438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.351441 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:17.351500 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.351502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.352828 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.352829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.354112 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.354114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.355234 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.355235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.355313 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.355318 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.355349 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.355401 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":66,"thread_start_us":86,"threads_started":1}
I20260808 07:11:17.355479 24609 tablet_service.cc:1467] Processing CreateTablet for tablet 56604a74402c493997e56d0b8e976183 (DEFAULT_TABLE table=test-table [id=7332c139473d400aadf1eedaae725b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:17.355921 25012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.350804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.22.2.131:38571)
0808 07:11:17.351233 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.351236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.351244 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:17.351276 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.351278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.351290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.351416 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:17.351596 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.352711 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.354213 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.355131 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.355667 (+ 536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.355733 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.355775 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.355814 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":109,"threads_started":1}
I20260808 07:11:17.355967 24609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56604a74402c493997e56d0b8e976183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:17.356460 24915 tablet_service.cc:1467] Processing CreateTablet for tablet 56604a74402c493997e56d0b8e976183 (DEFAULT_TABLE table=test-table [id=7332c139473d400aadf1eedaae725b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:17.356700 24915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56604a74402c493997e56d0b8e976183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:17.359079 25013 tablet_bootstrap.cc:492] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Bootstrap starting.
I20260808 07:11:17.359719 25014 tablet_bootstrap.cc:492] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Bootstrap starting.
I20260808 07:11:17.359812 25013 tablet_bootstrap.cc:654] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:17.359692 24765 tablet_service.cc:1467] Processing CreateTablet for tablet 56604a74402c493997e56d0b8e976183 (DEFAULT_TABLE table=test-table [id=7332c139473d400aadf1eedaae725b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:17.360049 24765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56604a74402c493997e56d0b8e976183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:17.360172 25013 log.cc:824] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:17.360785 25014 tablet_bootstrap.cc:654] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:17.361037 25013 tablet_bootstrap.cc:492] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: No bootstrap required, opened a new log
I20260808 07:11:17.361115 25013 ts_tablet_manager.cc:1397] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:17.361130 25014 log.cc:824] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:17.361956 25014 tablet_bootstrap.cc:492] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: No bootstrap required, opened a new log
I20260808 07:11:17.362041 25014 ts_tablet_manager.cc:1397] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:17.363283 25013 raft_consensus.cc:348] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.363466 25013 raft_consensus.cc:374] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:17.363507 25013 raft_consensus.cc:729] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5656022f91214a69b5c23b98bbe31009, State: Initialized, Role: FOLLOWER
I20260808 07:11:17.363446 25014 raft_consensus.cc:348] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.363667 25014 raft_consensus.cc:374] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:17.363708 25014 raft_consensus.cc:729] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915828ad95484fa99a950d110fdc8f16, State: Initialized, Role: FOLLOWER
I20260808 07:11:17.363701 25013 consensus_queue.cc:260] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.364022 25013 ts_tablet_manager.cc:1428] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:17.364132 25014 consensus_queue.cc:260] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.364192 25018 tablet_bootstrap.cc:492] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Bootstrap starting.
I20260808 07:11:17.364224 24681 heartbeater.cc:503] Master 127.22.2.190:37667 was elected leader, sending a full tablet report...
I20260808 07:11:17.364471 25014 ts_tablet_manager.cc:1428] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:17.364667 24989 heartbeater.cc:503] Master 127.22.2.190:37667 was elected leader, sending a full tablet report...
I20260808 07:11:17.364890 25018 tablet_bootstrap.cc:654] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:17.365259 25018 log.cc:824] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:17.366097 25018 tablet_bootstrap.cc:492] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: No bootstrap required, opened a new log
I20260808 07:11:17.366151 25018 ts_tablet_manager.cc:1397] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:17.367749 25018 raft_consensus.cc:348] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.367882 25018 raft_consensus.cc:374] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:17.367904 25018 raft_consensus.cc:729] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1c3af9dc427446fa78ebdef636c9052, State: Initialized, Role: FOLLOWER
I20260808 07:11:17.367985 25018 consensus_queue.cc:260] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.368176 25018 ts_tablet_manager.cc:1428] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:17.368252 24836 heartbeater.cc:503] Master 127.22.2.190:37667 was elected leader, sending a full tablet report...
W20260808 07:11:17.399993 24837 tablet.cc:2369] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:17.487293 24682 tablet.cc:2369] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:17.563251 24990 tablet.cc:2369] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:17.580600 25020 raft_consensus.cc:482] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:17.580770 25020 raft_consensus.cc:504] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.581179 25020 leader_election.cc:288] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231), e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125)
I20260808 07:11:17.583995 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.581450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:47891 (local address 127.22.2.130:44125)
0808 07:11:17.581497 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.581499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.581538 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:11:17.581570 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.581571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.581590 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.581662 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:17.581832 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.582307 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.583218 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.583642 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.583724 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.583733 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.583883 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.583919 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:17.583995 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.581373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44125 (local address 127.22.2.131:47891)
0808 07:11:17.581428 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.581442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.581493 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.581670 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.581671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.581673 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:17.581749 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.581751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.582396 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.582397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.583125 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.583127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.583720 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.583721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.583805 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.583808 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.583877 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.583919 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:11:17.584100 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.581487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:37865 (local address 127.22.2.129:37231)
0808 07:11:17.581531 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.581533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.581670 (+ 137us) server_negotiation.cc:407] Connection header received
0808 07:11:17.581727 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.581728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.581748 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.581820 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:17.581986 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.582454 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.583333 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.583761 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.583842 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.583850 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.583998 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.584034 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:11:17.584124 25023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.581397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37231 (local address 127.22.2.131:37865)
0808 07:11:17.581599 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.581607 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.581670 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.581824 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.581825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.581826 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:17.581915 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.581916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.582538 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.582539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.583236 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.583238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.583842 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.583843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.583925 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.583928 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.583994 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.584032 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":43,"thread_start_us":101,"threads_started":1}
I20260808 07:11:17.584450 24785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56604a74402c493997e56d0b8e976183" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052" is_pre_election: true
I20260808 07:11:17.584443 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56604a74402c493997e56d0b8e976183" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5656022f91214a69b5c23b98bbe31009" is_pre_election: true
I20260808 07:11:17.584618 24785 raft_consensus.cc:2454] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 0.
I20260808 07:11:17.584632 24632 raft_consensus.cc:2454] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 0.
I20260808 07:11:17.584880 24865 leader_election.cc:302] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915828ad95484fa99a950d110fdc8f16, e1c3af9dc427446fa78ebdef636c9052; no voters:
I20260808 07:11:17.585031 25020 raft_consensus.cc:2781] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:17.585093 25020 raft_consensus.cc:482] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:17.585112 25020 raft_consensus.cc:3037] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:17.586078 25020 raft_consensus.cc:504] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.586266 25020 leader_election.cc:288] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 election: Requested vote from peers 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231), e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125)
I20260808 07:11:17.586391 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56604a74402c493997e56d0b8e976183" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5656022f91214a69b5c23b98bbe31009"
I20260808 07:11:17.586424 24785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56604a74402c493997e56d0b8e976183" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052"
I20260808 07:11:17.586478 24632 raft_consensus.cc:3037] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:17.586493 24785 raft_consensus.cc:3037] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:17.587406 24632 raft_consensus.cc:2454] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 1.
I20260808 07:11:17.587406 24785 raft_consensus.cc:2454] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 1.
I20260808 07:11:17.587611 24867 leader_election.cc:302] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009, 915828ad95484fa99a950d110fdc8f16; no voters:
I20260808 07:11:17.587746 25020 raft_consensus.cc:2781] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:17.587898 25020 raft_consensus.cc:686] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 1 LEADER]: Becoming Leader. State: Replica: 915828ad95484fa99a950d110fdc8f16, State: Running, Role: LEADER
I20260808 07:11:17.588001 25020 consensus_queue.cc:237] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } }
I20260808 07:11:17.588837 24450 catalog_manager.cc:5495] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 915828ad95484fa99a950d110fdc8f16 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "915828ad95484fa99a950d110fdc8f16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:11:17 dist-test-slave-6k22 krb5kdc[24398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:17.659488 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.652447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37667 (local address 127.0.0.1:52368)
0808 07:11:17.652590 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.652600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.652738 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.652876 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.652877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.653243 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:17.653292 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.653294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.654057 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.654058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.654590 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.654592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.654995 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.654996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.655041 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.655633 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:17.655638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:17.657088 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:17.658525 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.658532 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.658534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.658774 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.658947 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:17.658948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:17.658948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:17.659004 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:17.659201 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:17.659207 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:17.659247 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:11:17.659299 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:11:17.659389 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260808 07:11:17.659590 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.652524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.22.2.190:37667)
0808 07:11:17.652597 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.652600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.652732 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:11:17.652789 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.652790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.652800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.652859 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.653397 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.653959 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.654688 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.654904 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.654986 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.657178 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:17.657179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:17.657180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:17.657185 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:17.658444 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.658839 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.658840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.658840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.658891 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:17.659055 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:17.659055 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:17.659056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:17.659149 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:17.659197 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:17.659329 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:11:17.659435 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":49}
I20260808 07:11:17.661720 24450 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:52368:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:11:17.661952 24450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:17.663503 24765 tablet_service.cc:1467] Processing CreateTablet for tablet af421126aba54b888016458ad1479beb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca883e3b771c4d249735c77606bc13d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:17.663532 24609 tablet_service.cc:1467] Processing CreateTablet for tablet af421126aba54b888016458ad1479beb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca883e3b771c4d249735c77606bc13d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:17.663614 24915 tablet_service.cc:1467] Processing CreateTablet for tablet af421126aba54b888016458ad1479beb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca883e3b771c4d249735c77606bc13d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:17.663740 24765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af421126aba54b888016458ad1479beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:17.663740 24609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af421126aba54b888016458ad1479beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:17.663769 24915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af421126aba54b888016458ad1479beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:17.664983 25018 tablet_bootstrap.cc:492] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: Bootstrap starting.
I20260808 07:11:17.665262 25013 tablet_bootstrap.cc:492] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: Bootstrap starting.
I20260808 07:11:17.665417 25018 tablet_bootstrap.cc:654] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:17.665683 25013 tablet_bootstrap.cc:654] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:17.665825 25014 tablet_bootstrap.cc:492] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: Bootstrap starting.
I20260808 07:11:17.666136 25018 tablet_bootstrap.cc:492] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: No bootstrap required, opened a new log
I20260808 07:11:17.666211 25018 ts_tablet_manager.cc:1397] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:17.666357 25013 tablet_bootstrap.cc:492] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: No bootstrap required, opened a new log
I20260808 07:11:17.666407 25013 ts_tablet_manager.cc:1397] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:17.666378 25018 raft_consensus.cc:348] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:17.666452 25018 raft_consensus.cc:374] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:17.666491 25018 raft_consensus.cc:729] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1c3af9dc427446fa78ebdef636c9052, State: Initialized, Role: FOLLOWER
I20260808 07:11:17.666543 25018 consensus_queue.cc:260] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:17.666569 25013 raft_consensus.cc:348] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:17.666666 25018 ts_tablet_manager.cc:1428] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:17.666639 25013 raft_consensus.cc:374] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:17.666706 25013 raft_consensus.cc:729] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5656022f91214a69b5c23b98bbe31009, State: Initialized, Role: FOLLOWER
I20260808 07:11:17.666764 25013 consensus_queue.cc:260] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:17.666810 25022 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:17.666874 25013 ts_tablet_manager.cc:1428] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:17.666906 25022 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:17.666951 25019 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:17.667054 25019 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:17.667485 25014 tablet_bootstrap.cc:654] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:17.668244 25014 tablet_bootstrap.cc:492] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: No bootstrap required, opened a new log
I20260808 07:11:17.668286 25014 ts_tablet_manager.cc:1397] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:17.668435 25014 raft_consensus.cc:348] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:17.668500 25014 raft_consensus.cc:374] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:17.668514 25014 raft_consensus.cc:729] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915828ad95484fa99a950d110fdc8f16, State: Initialized, Role: FOLLOWER
I20260808 07:11:17.668574 25014 consensus_queue.cc:260] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:17.668668 25014 ts_tablet_manager.cc:1428] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:17.668728 25020 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:17.668795 25020 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:18.060693 25020 consensus_queue.cc:1035] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:18.063179 25041 consensus_queue.cc:1035] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:18.081280 25019 raft_consensus.cc:482] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:18.081391 25019 raft_consensus.cc:504] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.081715 25019 leader_election.cc:288] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125), 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571)
I20260808 07:11:18.084982 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.081978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44125 (local address 127.22.2.129:44997)
0808 07:11:18.082149 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:18.082164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:18.082283 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:18.082436 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:18.082437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:18.082439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:18.082515 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.082517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.083172 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.083173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.084052 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.084054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.084706 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.084707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.084782 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.084786 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:18.084841 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:11:18.084888 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":39,"thread_start_us":94,"threads_started":1}
I20260808 07:11:18.085005 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.082042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:44997 (local address 127.22.2.130:44125)
0808 07:11:18.082126 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.082131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.082263 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:11:18.082345 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.082346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.082359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.082432 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:18.082593 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.083088 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.084146 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.084622 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.084710 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.084719 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:18.084873 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 07:11:18.084936 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:11:18.085071 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.082309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48183 (local address 127.22.2.131:38571)
0808 07:11:18.082437 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.082439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.082448 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:18.082504 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.082504 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.082516 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.082593 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:18.082785 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.083260 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.084314 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.084728 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.084791 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.084798 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:18.084935 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:11:18.084980 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260808 07:11:18.085064 25047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.082245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38571 (local address 127.22.2.129:48183)
0808 07:11:18.082365 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:18.082377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:18.082442 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:18.082611 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:18.082612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:18.082614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:18.082695 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.082697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.083348 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.083350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.084196 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.084199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.084789 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.084790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.084863 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.084865 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:18.084915 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:11:18.084943 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260808 07:11:18.085588 24785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af421126aba54b888016458ad1479beb" candidate_uuid: "5656022f91214a69b5c23b98bbe31009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052" is_pre_election: true
I20260808 07:11:18.085685 24785 raft_consensus.cc:2454] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5656022f91214a69b5c23b98bbe31009 in term 0.
I20260808 07:11:18.085604 24935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af421126aba54b888016458ad1479beb" candidate_uuid: "5656022f91214a69b5c23b98bbe31009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915828ad95484fa99a950d110fdc8f16" is_pre_election: true
I20260808 07:11:18.085762 24935 raft_consensus.cc:2454] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5656022f91214a69b5c23b98bbe31009 in term 0.
I20260808 07:11:18.085856 24553 leader_election.cc:302] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009, e1c3af9dc427446fa78ebdef636c9052; no voters:
I20260808 07:11:18.085990 25019 raft_consensus.cc:2781] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:18.086023 25019 raft_consensus.cc:482] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:18.086043 25019 raft_consensus.cc:3037] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:18.086781 25019 raft_consensus.cc:504] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.086928 25019 leader_election.cc:288] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [CANDIDATE]: Term 1 election: Requested vote from peers e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125), 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571)
I20260808 07:11:18.087110 24785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af421126aba54b888016458ad1479beb" candidate_uuid: "5656022f91214a69b5c23b98bbe31009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052"
I20260808 07:11:18.087201 24785 raft_consensus.cc:3037] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:18.087224 24935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af421126aba54b888016458ad1479beb" candidate_uuid: "5656022f91214a69b5c23b98bbe31009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915828ad95484fa99a950d110fdc8f16"
I20260808 07:11:18.087354 24935 raft_consensus.cc:3037] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:18.087960 24785 raft_consensus.cc:2454] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5656022f91214a69b5c23b98bbe31009 in term 1.
I20260808 07:11:18.088124 24553 leader_election.cc:302] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009, e1c3af9dc427446fa78ebdef636c9052; no voters:
I20260808 07:11:18.088229 25019 raft_consensus.cc:2781] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:18.088241 24935 raft_consensus.cc:2454] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5656022f91214a69b5c23b98bbe31009 in term 1.
I20260808 07:11:18.088367 25019 raft_consensus.cc:686] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [term 1 LEADER]: Becoming Leader. State: Replica: 5656022f91214a69b5c23b98bbe31009, State: Running, Role: LEADER
I20260808 07:11:18.088465 25019 consensus_queue.cc:237] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.088841 25050 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: TxnStatusTablet state changed. Reason: New leader 5656022f91214a69b5c23b98bbe31009. Latest consensus state: current_term: 1 leader_uuid: "5656022f91214a69b5c23b98bbe31009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:18.088961 25050 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:18.089195 25053 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:11:18.089306 25053 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:11:18.089170 24450 catalog_manager.cc:5495] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5656022f91214a69b5c23b98bbe31009 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "5656022f91214a69b5c23b98bbe31009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:18.109556 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.106389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37231 (local address 127.0.0.1:52998)
0808 07:11:18.106478 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:18.106494 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:18.106557 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:18.106858 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:18.106859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:18.106860 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:18.106911 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.106913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.107899 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.107900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.108489 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.108491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.109027 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.109029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.109089 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.109104 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:18.109400 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:18.109408 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 07:11:18.109448 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 07:11:18.109494 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 07:11:18.109575 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.106479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.22.2.129:37231)
0808 07:11:18.106663 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.106665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.106675 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:18.106715 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.106716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.106736 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.106826 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:18.107004 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.107821 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.108593 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.108935 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.109007 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.109209 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:18.109334 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:18.109415 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:18.109452 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:11:18.109473 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260808 07:11:18.111433 24785 raft_consensus.cc:1261] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Refusing update from remote peer 5656022f91214a69b5c23b98bbe31009: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:18.111593 24935 raft_consensus.cc:1261] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Refusing update from remote peer 5656022f91214a69b5c23b98bbe31009: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:18.111943 25050 consensus_queue.cc:1035] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:18.112087 25019 consensus_queue.cc:1035] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:18.113132 25022 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: TxnStatusTablet state changed. Reason: New leader 5656022f91214a69b5c23b98bbe31009. Latest consensus state: current_term: 1 leader_uuid: "5656022f91214a69b5c23b98bbe31009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:18.113233 25022 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:18.114246 25041 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: TxnStatusTablet state changed. Reason: New leader 5656022f91214a69b5c23b98bbe31009. Latest consensus state: current_term: 1 leader_uuid: "5656022f91214a69b5c23b98bbe31009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:18.114356 25041 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:18.114786 25042 mvcc.cc:205] Tried to move back new op lower bound from 7316164927941873664 to 7316164927850872832. Current Snapshot: MvccSnapshot[applied={T|T < 7316164927941873664}]
I20260808 07:11:18.115115 25019 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5656022f91214a69b5c23b98bbe31009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:18.115226 25019 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:18.115954 25041 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5656022f91214a69b5c23b98bbe31009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:18.116012 25040 mvcc.cc:205] Tried to move back new op lower bound from 7316164927941873664 to 7316164927850872832. Current Snapshot: MvccSnapshot[applied={T|T < 7316164927941873664}]
I20260808 07:11:18.116048 25041 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P 915828ad95484fa99a950d110fdc8f16: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:18.115116 25050 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5656022f91214a69b5c23b98bbe31009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:18.116220 25050 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P 5656022f91214a69b5c23b98bbe31009: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:18.116501 25045 mvcc.cc:205] Tried to move back new op lower bound from 7316164927941873664 to 7316164927850872832. Current Snapshot: MvccSnapshot[applied={T|T < 7316164927941873664}]
I20260808 07:11:18.116698 25022 tablet_replica.cc:439] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5656022f91214a69b5c23b98bbe31009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } } }
I20260808 07:11:18.116804 25022 tablet_replica.cc:442] T af421126aba54b888016458ad1479beb P e1c3af9dc427446fa78ebdef636c9052: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:18.119939 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.117264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.22.2.131:38571)
0808 07:11:18.117321 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.117323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.117460 (+ 137us) server_negotiation.cc:407] Connection header received
0808 07:11:18.117542 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.117542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.117561 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.117651 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:18.117803 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.118276 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.119090 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.119299 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.119375 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.119541 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:18.119702 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:18.119791 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:18.119830 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 07:11:18.119852 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:11:18.125443 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.122613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:54813 (local address 127.22.2.129:37231)
0808 07:11:18.122727 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.122729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.122738 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:18.122762 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.122762 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.122772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.122830 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:18.122976 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.123641 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.124762 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.124951 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.125014 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.125225 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:18.125252 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:18.125291 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:18.125370 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:11:18.125388 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 07:11:18.125765 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.122466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37231 (local address 127.22.2.131:54813)
0808 07:11:18.122615 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:18.122626 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:18.122672 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:18.122852 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:18.122853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:18.122856 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:18.122914 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.122916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.123723 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.123724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.124665 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.124668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.125033 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.125034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.125108 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.125124 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:18.125312 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:18.125314 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:18.125362 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:11:18.125405 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260808 07:11:18.136401 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.133630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48771 (local address 127.22.2.131:38571)
0808 07:11:18.133683 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.133684 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.133692 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:18.133760 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.133761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.133769 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.133813 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:18.133991 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.134450 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.135635 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.135867 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.135920 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.136110 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:18.136167 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:18.136209 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:18.136313 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:11:18.136335 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:11:18.136433 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.133465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38571 (local address 127.22.2.129:48771)
0808 07:11:18.133639 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:18.133651 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:18.133701 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:18.133816 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:18.133817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:18.133818 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:18.133879 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.133880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.134539 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.134540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.135515 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.135518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.135938 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.135939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.136008 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.136025 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:18.136230 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:18.136232 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:18.136275 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:11:18.136321 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260808 07:11:18.146641 24450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:18.146765 24450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:18.147677 24450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: Sending DeleteTablet for 3 replicas of tablet 56604a74402c493997e56d0b8e976183
I20260808 07:11:18.148052 24765 tablet_service.cc:1514] Processing DeleteTablet for tablet 56604a74402c493997e56d0b8e976183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:50690
I20260808 07:11:18.148120 24609 tablet_service.cc:1514] Processing DeleteTablet for tablet 56604a74402c493997e56d0b8e976183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:52988
I20260808 07:11:18.148355 25075 tablet_replica.cc:330] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: stopping tablet replica
I20260808 07:11:18.148473 25075 raft_consensus.cc:2229] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:18.148533 25075 raft_consensus.cc:2258] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:18.148523 24915 tablet_service.cc:1514] Processing DeleteTablet for tablet 56604a74402c493997e56d0b8e976183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:18 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60370
I20260808 07:11:18.149020 24450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:18.149161 24450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:18.150491 24765 tablet_service.cc:1467] Processing CreateTablet for tablet 1939a0d6f7bd47758ac584973b501150 (DEFAULT_TABLE table=test-table [id=482123bd55ba495b8c78e1665898bc63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:18.150625 24765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1939a0d6f7bd47758ac584973b501150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:18.152050 25018 tablet_bootstrap.cc:492] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: Bootstrap starting.
I20260808 07:11:18.152221 25075 ts_tablet_manager.cc:1905] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:18.152442 25018 tablet_bootstrap.cc:654] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:18.153218 24609 tablet_service.cc:1467] Processing CreateTablet for tablet 1939a0d6f7bd47758ac584973b501150 (DEFAULT_TABLE table=test-table [id=482123bd55ba495b8c78e1665898bc63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:18.153366 24609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1939a0d6f7bd47758ac584973b501150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:18.154256 25075 ts_tablet_manager.cc:1918] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:18.154326 25075 log.cc:1196] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/56604a74402c493997e56d0b8e976183
I20260808 07:11:18.154616 25075 ts_tablet_manager.cc:1939] T 56604a74402c493997e56d0b8e976183 P 5656022f91214a69b5c23b98bbe31009: Deleting consensus metadata
I20260808 07:11:18.154630 25013 tablet_bootstrap.cc:492] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: Bootstrap starting.
I20260808 07:11:18.154968 24915 tablet_service.cc:1467] Processing CreateTablet for tablet 1939a0d6f7bd47758ac584973b501150 (DEFAULT_TABLE table=test-table [id=482123bd55ba495b8c78e1665898bc63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:18.155098 24915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1939a0d6f7bd47758ac584973b501150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:18.155104 25013 tablet_bootstrap.cc:654] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:18.155285 24435 catalog_manager.cc:4841] TS 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231): tablet 56604a74402c493997e56d0b8e976183 (table test-table [id=7332c139473d400aadf1eedaae725b14]) successfully deleted
I20260808 07:11:18.156132 25076 tablet_replica.cc:330] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: stopping tablet replica
I20260808 07:11:18.156320 25076 raft_consensus.cc:2229] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:18.156409 25014 tablet_bootstrap.cc:492] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: Bootstrap starting.
I20260808 07:11:18.156519 25076 raft_consensus.cc:2258] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:18.156800 25014 tablet_bootstrap.cc:654] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:18.157482 25014 tablet_bootstrap.cc:492] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: No bootstrap required, opened a new log
I20260808 07:11:18.157531 25014 ts_tablet_manager.cc:1397] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:18.157667 25014 raft_consensus.cc:348] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.157749 25014 raft_consensus.cc:374] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:18.157775 25014 raft_consensus.cc:729] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915828ad95484fa99a950d110fdc8f16, State: Initialized, Role: FOLLOWER
I20260808 07:11:18.157816 25014 consensus_queue.cc:260] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.157897 25014 ts_tablet_manager.cc:1428] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:18.158289 25076 ts_tablet_manager.cc:1905] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:18.159704 25072 tablet_replica.cc:330] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: stopping tablet replica
I20260808 07:11:18.159852 25072 raft_consensus.cc:2229] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:18.159926 25072 raft_consensus.cc:2258] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:18.159981 25018 tablet_bootstrap.cc:492] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: No bootstrap required, opened a new log
I20260808 07:11:18.160043 25018 ts_tablet_manager.cc:1397] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260808 07:11:18.160055 25076 ts_tablet_manager.cc:1918] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:18.160120 25076 log.cc:1196] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/56604a74402c493997e56d0b8e976183
I20260808 07:11:18.160218 25018 raft_consensus.cc:348] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.160327 25018 raft_consensus.cc:374] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:18.160356 25018 raft_consensus.cc:729] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1c3af9dc427446fa78ebdef636c9052, State: Initialized, Role: FOLLOWER
I20260808 07:11:18.160398 25076 ts_tablet_manager.cc:1939] T 56604a74402c493997e56d0b8e976183 P 915828ad95484fa99a950d110fdc8f16: Deleting consensus metadata
I20260808 07:11:18.160401 25013 tablet_bootstrap.cc:492] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: No bootstrap required, opened a new log
I20260808 07:11:18.160449 25013 ts_tablet_manager.cc:1397] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260808 07:11:18.160416 25018 consensus_queue.cc:260] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.160527 25018 ts_tablet_manager.cc:1428] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:18.160547 25013 raft_consensus.cc:348] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.160615 25013 raft_consensus.cc:374] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:18.160604 25072 ts_tablet_manager.cc:1905] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:18.160637 25013 raft_consensus.cc:729] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5656022f91214a69b5c23b98bbe31009, State: Initialized, Role: FOLLOWER
I20260808 07:11:18.160681 25013 consensus_queue.cc:260] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.160789 25013 ts_tablet_manager.cc:1428] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:18.161412 24434 catalog_manager.cc:4841] TS 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571): tablet 56604a74402c493997e56d0b8e976183 (table test-table [id=7332c139473d400aadf1eedaae725b14]) successfully deleted
I20260808 07:11:18.165068 25072 ts_tablet_manager.cc:1918] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:18.165162 25072 log.cc:1196] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/56604a74402c493997e56d0b8e976183
I20260808 07:11:18.165437 25072 ts_tablet_manager.cc:1939] T 56604a74402c493997e56d0b8e976183 P e1c3af9dc427446fa78ebdef636c9052: Deleting consensus metadata
I20260808 07:11:18.165921 24434 catalog_manager.cc:4841] TS e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125): tablet 56604a74402c493997e56d0b8e976183 (table test-table [id=7332c139473d400aadf1eedaae725b14]) successfully deleted
I20260808 07:11:18.369850 25020 raft_consensus.cc:482] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:18.370003 25020 raft_consensus.cc:504] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.370231 25020 leader_election.cc:288] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231), e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125)
I20260808 07:11:18.370488 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1939a0d6f7bd47758ac584973b501150" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5656022f91214a69b5c23b98bbe31009" is_pre_election: true
I20260808 07:11:18.370519 24785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1939a0d6f7bd47758ac584973b501150" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052" is_pre_election: true
I20260808 07:11:18.370604 25050 raft_consensus.cc:482] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:18.370653 25050 raft_consensus.cc:504] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.370708 24785 raft_consensus.cc:2454] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 0.
I20260808 07:11:18.370723 24632 raft_consensus.cc:2454] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 0.
I20260808 07:11:18.370777 25050 leader_election.cc:288] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125), 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571)
I20260808 07:11:18.370913 24867 leader_election.cc:302] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009, 915828ad95484fa99a950d110fdc8f16; no voters:
I20260808 07:11:18.371027 25020 raft_consensus.cc:2781] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:18.371018 24785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1939a0d6f7bd47758ac584973b501150" candidate_uuid: "5656022f91214a69b5c23b98bbe31009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052" is_pre_election: true
I20260808 07:11:18.371083 25020 raft_consensus.cc:482] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:18.371070 24935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1939a0d6f7bd47758ac584973b501150" candidate_uuid: "5656022f91214a69b5c23b98bbe31009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915828ad95484fa99a950d110fdc8f16" is_pre_election: true
I20260808 07:11:18.371089 24785 raft_consensus.cc:2454] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5656022f91214a69b5c23b98bbe31009 in term 0.
I20260808 07:11:18.371119 25020 raft_consensus.cc:3037] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:18.371253 24553 leader_election.cc:302] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009, e1c3af9dc427446fa78ebdef636c9052; no voters:
I20260808 07:11:18.371377 25050 raft_consensus.cc:2781] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:18.371429 25050 raft_consensus.cc:482] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:18.371476 25050 raft_consensus.cc:3037] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:18.372028 25020 raft_consensus.cc:504] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.372191 25020 leader_election.cc:288] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 election: Requested vote from peers 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231), e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125)
I20260808 07:11:18.372205 24935 raft_consensus.cc:2379] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5656022f91214a69b5c23b98bbe31009 in current term 1: Already voted for candidate 915828ad95484fa99a950d110fdc8f16 in this term.
I20260808 07:11:18.372274 25050 raft_consensus.cc:504] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.372334 24785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1939a0d6f7bd47758ac584973b501150" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052"
I20260808 07:11:18.372399 24785 raft_consensus.cc:3037] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:18.372426 25050 leader_election.cc:288] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [CANDIDATE]: Term 1 election: Requested vote from peers e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125), 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571)
I20260808 07:11:18.372684 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1939a0d6f7bd47758ac584973b501150" candidate_uuid: "5656022f91214a69b5c23b98bbe31009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915828ad95484fa99a950d110fdc8f16"
I20260808 07:11:18.372684 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1939a0d6f7bd47758ac584973b501150" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5656022f91214a69b5c23b98bbe31009"
I20260808 07:11:18.372777 24941 raft_consensus.cc:2379] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5656022f91214a69b5c23b98bbe31009 in current term 1: Already voted for candidate 915828ad95484fa99a950d110fdc8f16 in this term.
I20260808 07:11:18.372809 24632 raft_consensus.cc:2379] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 915828ad95484fa99a950d110fdc8f16 in current term 1: Already voted for candidate 5656022f91214a69b5c23b98bbe31009 in this term.
I20260808 07:11:18.373080 24784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1939a0d6f7bd47758ac584973b501150" candidate_uuid: "5656022f91214a69b5c23b98bbe31009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052"
I20260808 07:11:18.373162 24785 raft_consensus.cc:2454] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 1.
I20260808 07:11:18.373318 24553 leader_election.cc:302] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009; no voters: 915828ad95484fa99a950d110fdc8f16, e1c3af9dc427446fa78ebdef636c9052
I20260808 07:11:18.373368 24865 leader_election.cc:302] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 915828ad95484fa99a950d110fdc8f16, e1c3af9dc427446fa78ebdef636c9052; no voters: 5656022f91214a69b5c23b98bbe31009
I20260808 07:11:18.373423 25050 raft_consensus.cc:2726] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 07:11:18.373474 25020 raft_consensus.cc:2781] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:18.373529 25020 raft_consensus.cc:686] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 1 LEADER]: Becoming Leader. State: Replica: 915828ad95484fa99a950d110fdc8f16, State: Running, Role: LEADER
I20260808 07:11:18.373596 25020 consensus_queue.cc:237] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:18.374207 24450 catalog_manager.cc:5495] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 915828ad95484fa99a950d110fdc8f16 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "915828ad95484fa99a950d110fdc8f16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } health_report { overall_health: HEALTHY } } }
I20260808 07:11:18.394155 24784 raft_consensus.cc:1261] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Refusing update from remote peer 915828ad95484fa99a950d110fdc8f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:18.394155 24632 raft_consensus.cc:1261] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Refusing update from remote peer 915828ad95484fa99a950d110fdc8f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:18.394469 25041 consensus_queue.cc:1035] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:18.394593 25020 consensus_queue.cc:1035] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:11:18.399379 25040 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:11:19.920641 24449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:19.920771 24449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:19.921653 24449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: Sending DeleteTablet for 3 replicas of tablet 1939a0d6f7bd47758ac584973b501150
I20260808 07:11:19.921958 24765 tablet_service.cc:1514] Processing DeleteTablet for tablet 1939a0d6f7bd47758ac584973b501150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:19 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:50690
I20260808 07:11:19.921962 24915 tablet_service.cc:1514] Processing DeleteTablet for tablet 1939a0d6f7bd47758ac584973b501150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:19 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60370
I20260808 07:11:19.922168 24609 tablet_service.cc:1514] Processing DeleteTablet for tablet 1939a0d6f7bd47758ac584973b501150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:19 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:52988
I20260808 07:11:19.922365 25107 tablet_replica.cc:330] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: stopping tablet replica
I20260808 07:11:19.922427 25107 raft_consensus.cc:2229] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:19.922472 25107 raft_consensus.cc:2258] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:19.922782 25107 ts_tablet_manager.cc:1905] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:19.925930 25105 tablet_replica.cc:330] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: stopping tablet replica
I20260808 07:11:19.926028 25105 raft_consensus.cc:2229] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:19.926190 25105 raft_consensus.cc:2258] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:19.926360 25106 tablet_replica.cc:330] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: stopping tablet replica
I20260808 07:11:19.926429 25106 raft_consensus.cc:2229] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:19.926476 25106 raft_consensus.cc:2258] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:19.926682 25105 ts_tablet_manager.cc:1905] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:19.927582 25107 ts_tablet_manager.cc:1918] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:11:19.927657 25107 log.cc:1196] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/1939a0d6f7bd47758ac584973b501150
I20260808 07:11:19.927919 25107 ts_tablet_manager.cc:1939] T 1939a0d6f7bd47758ac584973b501150 P 5656022f91214a69b5c23b98bbe31009: Deleting consensus metadata
I20260808 07:11:19.928339 24435 catalog_manager.cc:4841] TS 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231): tablet 1939a0d6f7bd47758ac584973b501150 (table test-table [id=482123bd55ba495b8c78e1665898bc63]) successfully deleted
I20260808 07:11:19.928586 25105 ts_tablet_manager.cc:1918] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:11:19.928648 25105 log.cc:1196] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/1939a0d6f7bd47758ac584973b501150
I20260808 07:11:19.928905 25105 ts_tablet_manager.cc:1939] T 1939a0d6f7bd47758ac584973b501150 P 915828ad95484fa99a950d110fdc8f16: Deleting consensus metadata
I20260808 07:11:19.929265 24434 catalog_manager.cc:4841] TS 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571): tablet 1939a0d6f7bd47758ac584973b501150 (table test-table [id=482123bd55ba495b8c78e1665898bc63]) successfully deleted
I20260808 07:11:19.929394 25106 ts_tablet_manager.cc:1905] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:19.930709 25106 ts_tablet_manager.cc:1918] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:11:19.930783 25106 log.cc:1196] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/1939a0d6f7bd47758ac584973b501150
I20260808 07:11:19.931056 25106 ts_tablet_manager.cc:1939] T 1939a0d6f7bd47758ac584973b501150 P e1c3af9dc427446fa78ebdef636c9052: Deleting consensus metadata
I20260808 07:11:19.931411 24434 catalog_manager.cc:4841] TS e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125): tablet 1939a0d6f7bd47758ac584973b501150 (table test-table [id=482123bd55ba495b8c78e1665898bc63]) successfully deleted
Aug 08 07:11:19 dist-test-slave-6k22 krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173079, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:11:19 dist-test-slave-6k22 krb5kdc[24398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173079, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:19.950259 25116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:19.942865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55062 (local address 127.22.2.190:37667)
0808 07:11:19.943009 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:19.943011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:19.943021 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:19.943105 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:19.943106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:19.943119 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:19.943182 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:19.943842 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.944367 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.945086 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.945299 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.945356 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:19.947806 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:19.947808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:19.947808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:19.947815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:19.949326 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:19.949614 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:19.949615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:19.949615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:19.949669 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:19.949820 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:19.949821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:19.949821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:19.949915 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:19.949985 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:19.950063 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:11:19.950123 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260808 07:11:19.951866 24449 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:19.952018 24449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:19.953462 24609 tablet_service.cc:1467] Processing CreateTablet for tablet 7c5bf32bb8c8469285d0de2dc133327a (DEFAULT_TABLE table=test-table [id=dca57a95762848d1ba70726a8ae7a3b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:19.953511 24765 tablet_service.cc:1467] Processing CreateTablet for tablet 7c5bf32bb8c8469285d0de2dc133327a (DEFAULT_TABLE table=test-table [id=dca57a95762848d1ba70726a8ae7a3b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:19.953552 24915 tablet_service.cc:1467] Processing CreateTablet for tablet 7c5bf32bb8c8469285d0de2dc133327a (DEFAULT_TABLE table=test-table [id=dca57a95762848d1ba70726a8ae7a3b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:19.953615 24609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c5bf32bb8c8469285d0de2dc133327a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:19.953661 24915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c5bf32bb8c8469285d0de2dc133327a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:19.953657 24765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c5bf32bb8c8469285d0de2dc133327a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:19.955116 25120 tablet_bootstrap.cc:492] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: Bootstrap starting.
I20260808 07:11:19.955124 25119 tablet_bootstrap.cc:492] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: Bootstrap starting.
I20260808 07:11:19.955164 25118 tablet_bootstrap.cc:492] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: Bootstrap starting.
I20260808 07:11:19.955502 25119 tablet_bootstrap.cc:654] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:19.955585 25118 tablet_bootstrap.cc:654] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:19.955614 25120 tablet_bootstrap.cc:654] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:19.956228 25119 tablet_bootstrap.cc:492] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: No bootstrap required, opened a new log
I20260808 07:11:19.956295 25119 ts_tablet_manager.cc:1397] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:19.956427 25119 raft_consensus.cc:348] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.956501 25119 raft_consensus.cc:374] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:19.956516 25119 raft_consensus.cc:729] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915828ad95484fa99a950d110fdc8f16, State: Initialized, Role: FOLLOWER
I20260808 07:11:19.956583 25119 consensus_queue.cc:260] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.956689 25119 ts_tablet_manager.cc:1428] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:19.956878 25120 tablet_bootstrap.cc:492] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: No bootstrap required, opened a new log
I20260808 07:11:19.956944 25120 ts_tablet_manager.cc:1397] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:11:19.957026 25118 tablet_bootstrap.cc:492] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: No bootstrap required, opened a new log
I20260808 07:11:19.957073 25118 ts_tablet_manager.cc:1397] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:19.957201 25118 raft_consensus.cc:348] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.957285 25118 raft_consensus.cc:374] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:19.957309 25118 raft_consensus.cc:729] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5656022f91214a69b5c23b98bbe31009, State: Initialized, Role: FOLLOWER
I20260808 07:11:19.957299 25120 raft_consensus.cc:348] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.957372 25118 consensus_queue.cc:260] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.957413 25120 raft_consensus.cc:374] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:19.957513 25118 ts_tablet_manager.cc:1428] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:19.957520 25120 raft_consensus.cc:729] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1c3af9dc427446fa78ebdef636c9052, State: Initialized, Role: FOLLOWER
I20260808 07:11:19.957600 25120 consensus_queue.cc:260] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.957913 25120 ts_tablet_manager.cc:1428] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:19.980871 25098 raft_consensus.cc:482] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:19.980989 25098 raft_consensus.cc:504] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.981205 25098 leader_election.cc:288] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231), e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125)
I20260808 07:11:19.981458 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c5bf32bb8c8469285d0de2dc133327a" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5656022f91214a69b5c23b98bbe31009" is_pre_election: true
I20260808 07:11:19.981495 24784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c5bf32bb8c8469285d0de2dc133327a" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052" is_pre_election: true
I20260808 07:11:19.981590 24632 raft_consensus.cc:2454] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 0.
I20260808 07:11:19.981614 24784 raft_consensus.cc:2454] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 0.
I20260808 07:11:19.981774 24867 leader_election.cc:302] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009, 915828ad95484fa99a950d110fdc8f16; no voters:
I20260808 07:11:19.981904 25098 raft_consensus.cc:2781] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:19.981954 25098 raft_consensus.cc:482] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:19.981971 25098 raft_consensus.cc:3037] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:19.982661 25098 raft_consensus.cc:504] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.982797 25098 leader_election.cc:288] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 election: Requested vote from peers 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231), e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125)
I20260808 07:11:19.982934 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c5bf32bb8c8469285d0de2dc133327a" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5656022f91214a69b5c23b98bbe31009"
I20260808 07:11:19.982933 24784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c5bf32bb8c8469285d0de2dc133327a" candidate_uuid: "915828ad95484fa99a950d110fdc8f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c3af9dc427446fa78ebdef636c9052"
I20260808 07:11:19.983016 24632 raft_consensus.cc:3037] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:19.983034 24784 raft_consensus.cc:3037] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:19.983716 24632 raft_consensus.cc:2454] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 1.
I20260808 07:11:19.983865 24784 raft_consensus.cc:2454] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915828ad95484fa99a950d110fdc8f16 in term 1.
I20260808 07:11:19.983891 24867 leader_election.cc:302] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009, 915828ad95484fa99a950d110fdc8f16; no voters:
I20260808 07:11:19.983968 25098 raft_consensus.cc:2781] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:19.984016 25098 raft_consensus.cc:686] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 1 LEADER]: Becoming Leader. State: Replica: 915828ad95484fa99a950d110fdc8f16, State: Running, Role: LEADER
I20260808 07:11:19.984063 25098 consensus_queue.cc:237] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:19.984664 24450 catalog_manager.cc:5495] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 915828ad95484fa99a950d110fdc8f16 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "915828ad95484fa99a950d110fdc8f16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } health_report { overall_health: HEALTHY } } }
I20260808 07:11:19.996032 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:19.993562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.22.2.131:38571)
0808 07:11:19.993720 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:19.993722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:19.993728 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:11:19.993746 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:19.993746 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:19.993756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:19.993843 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:19.993998 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.994497 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.995281 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.995495 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.995587 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:19.995756 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:19.995791 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:19.995855 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:19.995915 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:11:19.995939 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260808 07:11:19.998451 24632 raft_consensus.cc:1261] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Refusing update from remote peer 915828ad95484fa99a950d110fdc8f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:19.998461 24784 raft_consensus.cc:1261] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Refusing update from remote peer 915828ad95484fa99a950d110fdc8f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:19.998687 25098 consensus_queue.cc:1035] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:19.998782 25095 consensus_queue.cc:1035] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:20.008934 24449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:20.009035 24449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:20.009707 24449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: Sending DeleteTablet for 3 replicas of tablet 7c5bf32bb8c8469285d0de2dc133327a
I20260808 07:11:20.009943 24609 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c5bf32bb8c8469285d0de2dc133327a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:20 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:52988
I20260808 07:11:20.009974 24915 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c5bf32bb8c8469285d0de2dc133327a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:20 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60370
I20260808 07:11:20.009987 24765 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c5bf32bb8c8469285d0de2dc133327a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:20 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:50690
I20260808 07:11:20.010066 25105 tablet_replica.cc:330] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: stopping tablet replica
I20260808 07:11:20.010058 25107 tablet_replica.cc:330] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: stopping tablet replica
I20260808 07:11:20.010073 25106 tablet_replica.cc:330] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: stopping tablet replica
I20260808 07:11:20.010113 25105 raft_consensus.cc:2229] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:20.010119 25106 raft_consensus.cc:2229] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:20.010121 25107 raft_consensus.cc:2229] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:20.010157 25106 raft_consensus.cc:2258] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:20.010165 25105 raft_consensus.cc:2258] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:20.010169 25107 raft_consensus.cc:2258] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:20.010432 25106 ts_tablet_manager.cc:1905] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:20.010476 24449 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:20.010627 24449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:20.010651 25107 ts_tablet_manager.cc:1905] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:20.011950 24765 tablet_service.cc:1467] Processing CreateTablet for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab (DEFAULT_TABLE table=test-table [id=a36e8953f6e94a2e94542cce0062520b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:20.012072 24765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:20.012353 24915 tablet_service.cc:1467] Processing CreateTablet for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab (DEFAULT_TABLE table=test-table [id=a36e8953f6e94a2e94542cce0062520b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:20.012490 24915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:20.012651 25107 ts_tablet_manager.cc:1918] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:20.012660 25106 ts_tablet_manager.cc:1918] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:20.012706 25107 log.cc:1196] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/7c5bf32bb8c8469285d0de2dc133327a
I20260808 07:11:20.012719 25106 log.cc:1196] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/7c5bf32bb8c8469285d0de2dc133327a
I20260808 07:11:20.012948 25107 ts_tablet_manager.cc:1939] T 7c5bf32bb8c8469285d0de2dc133327a P 5656022f91214a69b5c23b98bbe31009: Deleting consensus metadata
I20260808 07:11:20.012957 25106 ts_tablet_manager.cc:1939] T 7c5bf32bb8c8469285d0de2dc133327a P e1c3af9dc427446fa78ebdef636c9052: Deleting consensus metadata
I20260808 07:11:20.013104 25105 ts_tablet_manager.cc:1905] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:20.013265 24434 catalog_manager.cc:4841] TS e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125): tablet 7c5bf32bb8c8469285d0de2dc133327a (table test-table [id=dca57a95762848d1ba70726a8ae7a3b2]) successfully deleted
I20260808 07:11:20.013401 24435 catalog_manager.cc:4841] TS 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231): tablet 7c5bf32bb8c8469285d0de2dc133327a (table test-table [id=dca57a95762848d1ba70726a8ae7a3b2]) successfully deleted
I20260808 07:11:20.013917 25119 tablet_bootstrap.cc:492] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: Bootstrap starting.
I20260808 07:11:20.014266 25119 tablet_bootstrap.cc:654] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:20.014520 25120 tablet_bootstrap.cc:492] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: Bootstrap starting.
I20260808 07:11:20.014596 25105 ts_tablet_manager.cc:1918] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:20.014652 25105 log.cc:1196] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/7c5bf32bb8c8469285d0de2dc133327a
I20260808 07:11:20.014803 25119 tablet_bootstrap.cc:492] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: No bootstrap required, opened a new log
I20260808 07:11:20.014878 25119 ts_tablet_manager.cc:1397] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:20.014906 25105 ts_tablet_manager.cc:1939] T 7c5bf32bb8c8469285d0de2dc133327a P 915828ad95484fa99a950d110fdc8f16: Deleting consensus metadata
I20260808 07:11:20.014943 25120 tablet_bootstrap.cc:654] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:20.014992 25119 raft_consensus.cc:348] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.015077 25119 raft_consensus.cc:374] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:20.015106 25119 raft_consensus.cc:729] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915828ad95484fa99a950d110fdc8f16, State: Initialized, Role: FOLLOWER
I20260808 07:11:20.015106 24609 tablet_service.cc:1467] Processing CreateTablet for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab (DEFAULT_TABLE table=test-table [id=a36e8953f6e94a2e94542cce0062520b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:20.015156 25119 consensus_queue.cc:260] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.015213 24609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:20.015255 25119 ts_tablet_manager.cc:1428] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:20.015657 24434 catalog_manager.cc:4841] TS 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571): tablet 7c5bf32bb8c8469285d0de2dc133327a (table test-table [id=dca57a95762848d1ba70726a8ae7a3b2]) successfully deleted
I20260808 07:11:20.015887 25120 tablet_bootstrap.cc:492] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: No bootstrap required, opened a new log
I20260808 07:11:20.015942 25120 ts_tablet_manager.cc:1397] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:20.016093 25120 raft_consensus.cc:348] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.016165 25120 raft_consensus.cc:374] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:20.016194 25120 raft_consensus.cc:729] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1c3af9dc427446fa78ebdef636c9052, State: Initialized, Role: FOLLOWER
I20260808 07:11:20.016239 25120 consensus_queue.cc:260] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.016342 25120 ts_tablet_manager.cc:1428] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:20.016510 25118 tablet_bootstrap.cc:492] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009: Bootstrap starting.
I20260808 07:11:20.016848 25118 tablet_bootstrap.cc:654] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:20.017452 25118 tablet_bootstrap.cc:492] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009: No bootstrap required, opened a new log
I20260808 07:11:20.017493 25118 ts_tablet_manager.cc:1397] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:20.017568 25118 raft_consensus.cc:348] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.017616 25118 raft_consensus.cc:374] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:20.017628 25118 raft_consensus.cc:729] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5656022f91214a69b5c23b98bbe31009, State: Initialized, Role: FOLLOWER
I20260808 07:11:20.017690 25118 consensus_queue.cc:260] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.017788 25118 ts_tablet_manager.cc:1428] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:20.172140 25124 raft_consensus.cc:482] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:20.172294 25124 raft_consensus.cc:504] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.172801 25124 leader_election.cc:288] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231), 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571)
I20260808 07:11:20.177337 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.173526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:56469 (local address 127.22.2.129:37231)
0808 07:11:20.173685 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:20.173688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:20.173696 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:20.173723 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:20.173724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:20.173736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:20.173839 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:20.174022 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.174852 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.176117 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.176868 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.176959 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.176969 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:20.177200 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 07:11:20.177246 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260808 07:11:20.177425 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.172996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37231 (local address 127.22.2.130:56469)
0808 07:11:20.173172 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:20.173186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:20.173251 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:20.173851 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:20.173852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:20.173854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:20.173927 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.173929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.174950 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.174951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.176001 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.176003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.176975 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.176977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.177091 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.177096 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:20.177182 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:11:20.177256 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":33,"thread_start_us":68,"threads_started":1}
I20260808 07:11:20.177798 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.173156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:50797 (local address 127.22.2.131:38571)
0808 07:11:20.173291 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:20.173294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:20.173344 (+ 50us) server_negotiation.cc:407] Connection header received
0808 07:11:20.173477 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:20.173478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:20.173494 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:20.173561 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:20.173823 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.174776 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.176706 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.177322 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.177399 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.177410 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:20.177671 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 07:11:20.177722 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260808 07:11:20.177969 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccc70321e1e74c3c95e58d1d1f4ff0ab" candidate_uuid: "e1c3af9dc427446fa78ebdef636c9052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5656022f91214a69b5c23b98bbe31009" is_pre_election: true
I20260808 07:11:20.178072 24632 raft_consensus.cc:2454] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1c3af9dc427446fa78ebdef636c9052 in term 0.
I20260808 07:11:20.178083 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.173041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38571 (local address 127.22.2.130:50797)
0808 07:11:20.173247 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:20.173259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:20.173379 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:20.173598 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:20.173599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:20.173602 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:20.173719 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.173722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.174889 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.174890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.176604 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.176608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.177479 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.177481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.177556 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.177560 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:20.177897 (+ 337us) client_negotiation.cc:240] Negotiation successful
0808 07:11:20.177952 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":108,"threads_started":1}
I20260808 07:11:20.178282 24712 leader_election.cc:302] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5656022f91214a69b5c23b98bbe31009, e1c3af9dc427446fa78ebdef636c9052; no voters:
I20260808 07:11:20.178380 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccc70321e1e74c3c95e58d1d1f4ff0ab" candidate_uuid: "e1c3af9dc427446fa78ebdef636c9052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915828ad95484fa99a950d110fdc8f16" is_pre_election: true
I20260808 07:11:20.178470 24941 raft_consensus.cc:2454] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1c3af9dc427446fa78ebdef636c9052 in term 0.
I20260808 07:11:20.178545 25124 raft_consensus.cc:2781] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:20.178588 25124 raft_consensus.cc:482] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:20.178617 25124 raft_consensus.cc:3037] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:20.179855 25124 raft_consensus.cc:504] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.180042 25124 leader_election.cc:288] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [CANDIDATE]: Term 1 election: Requested vote from peers 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231), 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571)
I20260808 07:11:20.180184 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccc70321e1e74c3c95e58d1d1f4ff0ab" candidate_uuid: "e1c3af9dc427446fa78ebdef636c9052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5656022f91214a69b5c23b98bbe31009"
I20260808 07:11:20.180277 24632 raft_consensus.cc:3037] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:20.180275 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccc70321e1e74c3c95e58d1d1f4ff0ab" candidate_uuid: "e1c3af9dc427446fa78ebdef636c9052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915828ad95484fa99a950d110fdc8f16"
I20260808 07:11:20.180358 24941 raft_consensus.cc:3037] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:20.181185 24632 raft_consensus.cc:2454] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1c3af9dc427446fa78ebdef636c9052 in term 1.
I20260808 07:11:20.181479 24941 raft_consensus.cc:2454] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1c3af9dc427446fa78ebdef636c9052 in term 1.
I20260808 07:11:20.182067 24711 leader_election.cc:302] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915828ad95484fa99a950d110fdc8f16, e1c3af9dc427446fa78ebdef636c9052; no voters:
I20260808 07:11:20.182200 25124 raft_consensus.cc:2781] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:20.182387 25124 raft_consensus.cc:686] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 1 LEADER]: Becoming Leader. State: Replica: e1c3af9dc427446fa78ebdef636c9052, State: Running, Role: LEADER
I20260808 07:11:20.182458 25124 consensus_queue.cc:237] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } }
I20260808 07:11:20.183233 24449 catalog_manager.cc:5495] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "e1c3af9dc427446fa78ebdef636c9052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1c3af9dc427446fa78ebdef636c9052" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:20.199331 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.196945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.22.2.130:44125)
0808 07:11:20.197119 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:20.197121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:20.197129 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:20.197196 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:20.197197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:20.197217 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:20.197286 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:20.197381 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.197903 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.198592 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.198866 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.198942 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.199042 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:20.199158 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:20.199212 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:20.199246 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 07:11:20.199261 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260808 07:11:20.204124 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.201807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:52843 (local address 127.22.2.129:37231)
0808 07:11:20.201846 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:20.201848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:20.202056 (+ 208us) server_negotiation.cc:407] Connection header received
0808 07:11:20.202087 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:20.202088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:20.202106 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:20.202163 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:20.202281 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.202742 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.203451 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.203727 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.203786 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.203948 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:20.203974 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:20.204012 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:20.204046 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 07:11:20.204064 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:11:20.204140 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.201730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37231 (local address 127.22.2.130:52843)
0808 07:11:20.201901 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:20.201911 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:20.201975 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:20.202176 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:20.202177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:20.202178 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:20.202238 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.202239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.202799 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.202799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.203380 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.203381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.203814 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.203815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.203863 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.203877 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:20.204013 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:20.204014 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:11:20.204043 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 07:11:20.204075 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260808 07:11:20.206712 24632 raft_consensus.cc:1261] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 5656022f91214a69b5c23b98bbe31009 [term 1 FOLLOWER]: Refusing update from remote peer e1c3af9dc427446fa78ebdef636c9052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:20.206784 24941 raft_consensus.cc:1261] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Refusing update from remote peer e1c3af9dc427446fa78ebdef636c9052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:20.207037 25138 consensus_queue.cc:1035] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5656022f91214a69b5c23b98bbe31009" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:20.207206 25124 consensus_queue.cc:1035] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915828ad95484fa99a950d110fdc8f16" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:20.214891 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.212538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44125 (local address 127.22.2.129:54565)
0808 07:11:20.212676 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:20.212687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:20.212748 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:20.212866 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:20.212866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:20.212867 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:20.212910 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.212911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.213453 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.213454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.214084 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.214086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.214462 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.214463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.214520 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.214524 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:20.214719 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:20.214721 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:20.214773 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:11:20.214817 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":95,"threads_started":1}
I20260808 07:11:20.214876 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.212597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:54565 (local address 127.22.2.130:44125)
0808 07:11:20.212684 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:20.212686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:20.212732 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:11:20.212812 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:20.212813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:20.212821 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:20.212858 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:20.212945 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.213395 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.214174 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.214380 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.214452 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.214603 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:20.214641 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:20.214714 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:20.214778 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:11:20.214801 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 07:11:20.215350 25087 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:11:21.739447 24450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:21.739581 24450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:21.740543 24450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c556c1164e324d0d9b778c6e748bb125: Sending DeleteTablet for 3 replicas of tablet ccc70321e1e74c3c95e58d1d1f4ff0ab
I20260808 07:11:21.740855 24765 tablet_service.cc:1514] Processing DeleteTablet for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:21 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:50690
I20260808 07:11:21.740877 24915 tablet_service.cc:1514] Processing DeleteTablet for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:21 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60370
I20260808 07:11:21.741082 25176 tablet_replica.cc:330] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: stopping tablet replica
I20260808 07:11:21.741056 24609 tablet_service.cc:1514] Processing DeleteTablet for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:21 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:52988
I20260808 07:11:21.741184 25176 raft_consensus.cc:2229] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:21.741205 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 24534
I20260808 07:11:21.742583 25176 raft_consensus.cc:2258] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:21.742926 25176 ts_tablet_manager.cc:1905] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:21.744939 25176 ts_tablet_manager.cc:1918] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:11:21.745025 25176 log.cc:1196] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/ccc70321e1e74c3c95e58d1d1f4ff0ab
I20260808 07:11:21.745277 25176 ts_tablet_manager.cc:1939] T ccc70321e1e74c3c95e58d1d1f4ff0ab P e1c3af9dc427446fa78ebdef636c9052: Deleting consensus metadata
I20260808 07:11:21.745438 25175 tablet_replica.cc:330] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: stopping tablet replica
I20260808 07:11:21.745512 25175 raft_consensus.cc:2229] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:21.745561 25175 raft_consensus.cc:2258] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:21.746061 25175 ts_tablet_manager.cc:1905] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:21.748339 25175 ts_tablet_manager.cc:1918] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:11:21.748448 25175 log.cc:1196] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/ccc70321e1e74c3c95e58d1d1f4ff0ab
I20260808 07:11:21.748698 25175 ts_tablet_manager.cc:1939] T ccc70321e1e74c3c95e58d1d1f4ff0ab P 915828ad95484fa99a950d110fdc8f16: Deleting consensus metadata
I20260808 07:11:21.748993 24434 catalog_manager.cc:4841] TS e1c3af9dc427446fa78ebdef636c9052 (127.22.2.130:44125): tablet ccc70321e1e74c3c95e58d1d1f4ff0ab (table test-table [id=a36e8953f6e94a2e94542cce0062520b]) successfully deleted
I20260808 07:11:21.749087 24434 catalog_manager.cc:4841] TS 915828ad95484fa99a950d110fdc8f16 (127.22.2.131:38571): tablet ccc70321e1e74c3c95e58d1d1f4ff0ab (table test-table [id=a36e8953f6e94a2e94542cce0062520b]) successfully deleted
W20260808 07:11:21.754459 24435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:37231 (error 108)
I20260808 07:11:21.754817 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 24690
I20260808 07:11:21.756439 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:21.754975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37231 (local address 127.0.0.1:55350)
0808 07:11:21.756326 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:21.756386 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1323,"thread_start_us":72,"threads_started":1}
W20260808 07:11:21.756589 24435 catalog_manager.cc:4584] TS 5656022f91214a69b5c23b98bbe31009 (127.22.2.129:37231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ccc70321e1e74c3c95e58d1d1f4ff0ab: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37231: connect: Connection refused (error 111)
I20260808 07:11:21.764411 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 24846
I20260808 07:11:21.772692 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 24415
2026-08-08T07:11:21Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6351 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:21 dist-test-slave-6k22 krb5kdc[25183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:21 dist-test-slave-6k22 krb5kdc[25183](info): set up 2 sockets
Aug 08 07:11:21 dist-test-slave-6k22 krb5kdc[25183](info): commencing operation
krb5kdc: starting...
W20260808 07:11:22.899300 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:22 dist-test-slave-6k22 krb5kdc[25183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:23.027194 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35269
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:23.111785 25196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:23.111985 25196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:23.112007 25196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:23.113174 25196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:23.113219 25196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:23.113233 25196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:23.113246 25196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:23.113286 25196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:23.114835 25196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:23.115088 25196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:23.115330 25196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:23.120357 25206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:23.120342 25204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:23.120342 25203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:23.120709 25196 server_base.cc:1034] running on GCE node
I20260808 07:11:23.120991 25196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:23.121273 25196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:23.122423 25196 hybrid_clock.cc:648] HybridClock initialized: now 1786173083122421 us; error 60 us; skew 500 ppm
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173083, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:23.130574 25196 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:23.131171 25196 webserver.cc:460] Webserver started at http://127.22.2.190:44079/ using document root <none> and password file <none>
I20260808 07:11:23.131425 25196 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:23.131500 25196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:23.131747 25196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:23.132923 25196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "2b9924eccbc14099862417803c15433c"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "e14b2bfa6b5824e601cd38dbddf6b171"
server_key_iv: "b0e0301dbf8ba6c19e94a0f014121cf9"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.133334 25196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "2b9924eccbc14099862417803c15433c"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "e14b2bfa6b5824e601cd38dbddf6b171"
server_key_iv: "b0e0301dbf8ba6c19e94a0f014121cf9"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.134789 25196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:23.135633 25215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.135823 25196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:23.135949 25196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "2b9924eccbc14099862417803c15433c"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "e14b2bfa6b5824e601cd38dbddf6b171"
server_key_iv: "b0e0301dbf8ba6c19e94a0f014121cf9"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.136029 25196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:23.147425 25196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:23.149329 25196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:23.149508 25196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:23.154098 25196 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35269
I20260808 07:11:23.154127 25274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35269 every 8 connection(s)
I20260808 07:11:23.154508 25196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:23.155385 25275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:23.157676 25275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c: Bootstrap starting.
I20260808 07:11:23.158309 25275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:23.158661 25275 log.cc:824] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:23.159384 25275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c: No bootstrap required, opened a new log
I20260808 07:11:23.161020 25275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9924eccbc14099862417803c15433c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35269 } }
I20260808 07:11:23.161157 25275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:23.161180 25275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9924eccbc14099862417803c15433c, State: Initialized, Role: FOLLOWER
I20260808 07:11:23.161312 25275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9924eccbc14099862417803c15433c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35269 } }
I20260808 07:11:23.161393 25275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:23.161417 25275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:23.161446 25275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:23.161942 25275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9924eccbc14099862417803c15433c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35269 } }
I20260808 07:11:23.162027 25275 leader_election.cc:302] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b9924eccbc14099862417803c15433c; no voters:
I20260808 07:11:23.162143 25275 leader_election.cc:288] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:23.162201 25280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:23.162400 25280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [term 1 LEADER]: Becoming Leader. State: Replica: 2b9924eccbc14099862417803c15433c, State: Running, Role: LEADER
I20260808 07:11:23.162437 25275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:23.162578 25280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9924eccbc14099862417803c15433c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35269 } }
I20260808 07:11:23.162801 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 25196
I20260808 07:11:23.162905 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:23.163112 22538 external_mini_cluster.cc:1428] Setting key cb6101d041720ecc2be712f1f7dc9b5b
I20260808 07:11:23.163305 25283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b9924eccbc14099862417803c15433c. Latest consensus state: current_term: 1 leader_uuid: "2b9924eccbc14099862417803c15433c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9924eccbc14099862417803c15433c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35269 } } }
I20260808 07:11:23.163403 25283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:23.163666 25281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b9924eccbc14099862417803c15433c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9924eccbc14099862417803c15433c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35269 } } }
I20260808 07:11:23.163839 25281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:23.163971 25294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:23.164697 25294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:23.166138 25294 catalog_manager.cc:1347] Generated new cluster ID: f2f5d917a57346bdb2b53d4ecf3e9ea2
I20260808 07:11:23.166193 25294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:23.171725 25294 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:23.172390 25294 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:23.174817 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.163845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.22.2.190:35269)
0808 07:11:23.164232 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:23.164242 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:23.164274 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:11:23.164395 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:23.164403 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:23.164490 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:23.164627 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:23.165527 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.166742 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.167530 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.167990 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.168121 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.170515 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:23.170519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:23.170519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:23.170537 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:23.173185 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:23.173557 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:23.173560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:23.173560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:23.173664 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:23.173882 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:23.173882 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:23.173883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:23.174105 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:23.174272 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:23.174372 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:11:23.174640 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":72,"threads_started":1}
I20260808 07:11:23.178458 25294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b9924eccbc14099862417803c15433c: Generated new TSK 0
I20260808 07:11:23.178709 25294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:23.210801 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35269
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:23.296103 25308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:23.296356 25308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:23.296401 25308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:23.298225 25308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:23.298359 25308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:23.298413 25308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:23.300041 25308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:23.300329 25308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:23.300581 25308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:23.305949 25313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:23.306025 25314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:23.306041 25308 server_base.cc:1034] running on GCE node
W20260808 07:11:23.305949 25316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:23.306406 25308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:23.306667 25308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:23.307817 25308 hybrid_clock.cc:648] HybridClock initialized: now 1786173083307794 us; error 36 us; skew 500 ppm
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173083, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:23.310382 25308 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:23.310844 25308 webserver.cc:460] Webserver started at http://127.22.2.129:33503/ using document root <none> and password file <none>
I20260808 07:11:23.311075 25308 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:23.311121 25308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:23.311280 25308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:23.312433 25308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "2ebf6ab6976f4dc48f6a0b6517e8970f"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "d0026940a1f97f07ca4f59cfcd860188"
server_key_iv: "dd3628b85480291a2ab57185f737dccc"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.312800 25308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "2ebf6ab6976f4dc48f6a0b6517e8970f"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "d0026940a1f97f07ca4f59cfcd860188"
server_key_iv: "dd3628b85480291a2ab57185f737dccc"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.314188 25308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:11:23.315011 25324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.315213 25308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:23.315310 25308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "2ebf6ab6976f4dc48f6a0b6517e8970f"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "d0026940a1f97f07ca4f59cfcd860188"
server_key_iv: "dd3628b85480291a2ab57185f737dccc"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.315366 25308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:23.333122 25308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:23.335489 25308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:23.335688 25308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:23.335927 25308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:23.336268 25308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:23.336313 25308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.336355 25308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:23.336381 25308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.342437 25308 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:34185
I20260808 07:11:23.342550 25454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:34185 every 8 connection(s)
I20260808 07:11:23.342868 25308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:23.347179 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 25308
I20260808 07:11:23.347293 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:23.347486 22538 external_mini_cluster.cc:1428] Setting key fa28436a8bd3552de06573e5e7ac2ba2
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173083, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:23.353005 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.343620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:40285 (local address 127.22.2.190:35269)
0808 07:11:23.343680 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:23.343682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:23.343924 (+ 242us) server_negotiation.cc:407] Connection header received
0808 07:11:23.344394 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:23.344395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:23.344405 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:23.344458 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:23.345375 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.345930 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.346917 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.347123 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.347171 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.349909 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:23.349910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:23.349911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:23.349920 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:23.351346 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:23.351887 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:23.351888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:23.351889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:23.351960 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:23.352305 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:23.352306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:23.352306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:23.352433 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:23.352495 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:23.352853 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 07:11:23.352936 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:11:23.353317 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.343619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35269 (local address 127.22.2.129:40285)
0808 07:11:23.343776 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:23.343835 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:23.344314 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:23.344510 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:23.344513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:23.345091 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:23.345287 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.345291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.346017 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.346018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.346841 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.346843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.347208 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.347210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.347368 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.348178 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:23.348189 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:23.349805 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:23.351455 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:23.351463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:23.351465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:23.351802 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:23.352042 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:23.352043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:23.352043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:23.352190 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:23.352504 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:23.352511 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:23.352683 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 07:11:23.352791 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:11:23.353012 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260808 07:11:23.353940 25455 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35269
I20260808 07:11:23.354051 25455 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:23.354264 25455 heartbeater.cc:511] Master 127.22.2.190:35269 requested a full tablet report, sending...
I20260808 07:11:23.354846 25235 ts_manager.cc:194] Registered new tserver with Master: 2ebf6ab6976f4dc48f6a0b6517e8970f (127.22.2.129:34185)
I20260808 07:11:23.355474 25235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:40285
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:23.378743 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35269
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:23.464020 25460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:23.464284 25460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:23.464313 25460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:23.465503 25460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:23.465546 25460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:23.465574 25460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:23.467061 25460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:23.467326 25460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:23.467587 25460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:23.472909 25460 server_base.cc:1034] running on GCE node
W20260808 07:11:23.472929 25467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:23.473043 25469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:23.472930 25465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:23.473323 25460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:23.473583 25460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:23.474737 25460 hybrid_clock.cc:648] HybridClock initialized: now 1786173083474714 us; error 40 us; skew 500 ppm
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173083, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:23.477301 25460 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:23.477759 25460 webserver.cc:460] Webserver started at http://127.22.2.130:35397/ using document root <none> and password file <none>
I20260808 07:11:23.478013 25460 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:23.478063 25460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:23.478192 25460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:23.479256 25460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "95c0ff186788461c9980beec7c1d20c8"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "9fcf6d372d14cfe61afbb3402bd25196"
server_key_iv: "5ca1ad63be87f7753afefbde3d82286d"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.479667 25460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "95c0ff186788461c9980beec7c1d20c8"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "9fcf6d372d14cfe61afbb3402bd25196"
server_key_iv: "5ca1ad63be87f7753afefbde3d82286d"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.481033 25460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:23.481856 25476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.482061 25460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:23.482163 25460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "95c0ff186788461c9980beec7c1d20c8"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "9fcf6d372d14cfe61afbb3402bd25196"
server_key_iv: "5ca1ad63be87f7753afefbde3d82286d"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.482241 25460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:23.505112 25460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:23.507325 25460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:23.507491 25460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:23.507754 25460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:23.508101 25460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:23.508158 25460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.508204 25460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:23.508229 25460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.514240 25460 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:43813
I20260808 07:11:23.514314 25604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:43813 every 8 connection(s)
I20260808 07:11:23.514726 25460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:23.515120 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 25460
I20260808 07:11:23.515234 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:23.515501 22538 external_mini_cluster.cc:1428] Setting key b5e5471d073ee5cc30d1996a01f87bbc
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173083, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 07:11:23.526275 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.515538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:51411 (local address 127.22.2.190:35269)
0808 07:11:23.515621 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:23.515623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:23.515889 (+ 266us) server_negotiation.cc:407] Connection header received
0808 07:11:23.516542 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:23.516543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:23.516557 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:23.516603 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:23.518137 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.518827 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.520038 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.520258 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.520354 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.523062 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:23.523063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:23.523064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:23.523073 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:23.524663 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:23.525144 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:23.525145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:23.525145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:23.525227 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:23.525509 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:23.525510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:23.525511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:23.525647 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:23.525729 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:23.526057 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 07:11:23.526150 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:11:23.526510 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.515443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35269 (local address 127.22.2.130:51411)
0808 07:11:23.515712 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:23.515768 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:23.516453 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:23.516764 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:23.516769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:23.517664 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:23.517952 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.517959 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.518935 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.518936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.519923 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.519926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.520368 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.520370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.520513 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.521118 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:23.521127 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:23.522944 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:23.524779 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:23.524788 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:23.524790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:23.525045 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:23.525288 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:23.525289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:23.525289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:23.525413 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:23.525734 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:23.525742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:23.525900 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 07:11:23.526019 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:11:23.526244 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":85,"threads_started":1}
I20260808 07:11:23.527061 25605 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35269
I20260808 07:11:23.527163 25605 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:23.527338 25605 heartbeater.cc:511] Master 127.22.2.190:35269 requested a full tablet report, sending...
I20260808 07:11:23.527866 25235 ts_manager.cc:194] Registered new tserver with Master: 95c0ff186788461c9980beec7c1d20c8 (127.22.2.130:43813)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 07:11:23.528491 25235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:51411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:23.544697 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35269
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:23.632532 25611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:23.632807 25611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:23.632841 25611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:23.634043 25611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:23.634087 25611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:23.634116 25611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:23.635636 25611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:23.635864 25611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:23.636063 25611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:23.641467 25616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:23.641429 25618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:23.641497 25620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:23.641634 25611 server_base.cc:1034] running on GCE node
I20260808 07:11:23.641945 25611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:23.642233 25611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:23.643396 25611 hybrid_clock.cc:648] HybridClock initialized: now 1786173083643375 us; error 40 us; skew 500 ppm
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173083, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:23.646016 25611 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:23.646474 25611 webserver.cc:460] Webserver started at http://127.22.2.131:40775/ using document root <none> and password file <none>
I20260808 07:11:23.646670 25611 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:23.646711 25611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:23.646800 25611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:23.647893 25611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "7b50e7ed368649f6a8e1907a111cc1dc"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "d1bfc6940e773e27a9c2e5fa58b4e385"
server_key_iv: "b3f24de40c7165b13155b8428c72b03c"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.648203 25611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "7b50e7ed368649f6a8e1907a111cc1dc"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "d1bfc6940e773e27a9c2e5fa58b4e385"
server_key_iv: "b3f24de40c7165b13155b8428c72b03c"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.649482 25611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:23.650218 25628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.650408 25611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:23.650523 25611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "7b50e7ed368649f6a8e1907a111cc1dc"
format_stamp: "Formatted at 2026-08-08 07:11:23 on dist-test-slave-6k22"
server_key: "d1bfc6940e773e27a9c2e5fa58b4e385"
server_key_iv: "b3f24de40c7165b13155b8428c72b03c"
server_key_version: "encryptionkey@0"
I20260808 07:11:23.650599 25611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:23.671217 25611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:23.673741 25611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:23.673918 25611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:23.674162 25611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:23.674499 25611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:23.674561 25611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.674605 25611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:23.674630 25611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:23.680738 25611 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:44035
I20260808 07:11:23.680831 25757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:44035 every 8 connection(s)
I20260808 07:11:23.681209 25611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173083, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:23.690676 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.681991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:37043 (local address 127.22.2.190:35269)
0808 07:11:23.682070 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:23.682072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:23.682251 (+ 179us) server_negotiation.cc:407] Connection header received
0808 07:11:23.682753 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:23.682754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:23.682765 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:23.682824 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:23.683824 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.684312 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.685408 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.685595 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.685647 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.687962 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:23.687970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:23.687971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:23.687979 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:23.689373 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:23.689807 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:23.689808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:23.689809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:23.689868 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:23.690097 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:23.690098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:23.690098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:23.690223 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:23.690278 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:23.690518 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 07:11:23.690586 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:11:23.690831 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 25611
I20260808 07:11:23.690934 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:23.691049 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.681950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35269 (local address 127.22.2.131:37043)
0808 07:11:23.682125 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:23.682162 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:23.682668 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:23.682881 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:23.682884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:23.683456 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:23.683739 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.683744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.684393 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.684394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.685339 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.685342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.685730 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.685731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.685834 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.686362 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:23.686371 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:23.687848 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:23.689463 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:23.689469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:23.689471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:23.689718 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:23.689934 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:23.689935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:23.689935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:23.690040 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:23.690281 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:23.690285 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:23.690405 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 07:11:23.690487 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:11:23.690633 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260808 07:11:23.691147 22538 external_mini_cluster.cc:1428] Setting key fb95ecbe245d140d83e8cfd0729ec9af
I20260808 07:11:23.691694 25758 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35269
I20260808 07:11:23.691802 25758 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:23.692016 25758 heartbeater.cc:511] Master 127.22.2.190:35269 requested a full tablet report, sending...
I20260808 07:11:23.692451 25234 ts_manager.cc:194] Registered new tserver with Master: 7b50e7ed368649f6a8e1907a111cc1dc (127.22.2.131:44035)
I20260808 07:11:23.692885 25234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:37043
I20260808 07:11:23.703313 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:11:23.721678 25297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:23.720638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.22.2.190:35269)
0808 07:11:23.720712 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:23.720714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:23.720836 (+ 122us) server_negotiation.cc:407] Connection header received
0808 07:11:23.720896 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:23.720897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:23.720908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:23.720956 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:23.721560 (+ 604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33198: BlockingRecv error: recv got EOF from 127.0.0.1:33198 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:11:23.722509 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 25308
I20260808 07:11:23.729969 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 25460
I20260808 07:11:23.738368 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 25611
I20260808 07:11:23.745668 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 25196
2026-08-08T07:11:23Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1970 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25770](info): set up 2 sockets
Aug 08 07:11:23 dist-test-slave-6k22 krb5kdc[25770](info): commencing operation
krb5kdc: starting...
W20260808 07:11:24.926151 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:25 dist-test-slave-6k22 krb5kdc[25770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:25.053903 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46773
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:46773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:25.138095 25783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:25.138373 25783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:25.138415 25783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:25.139623 25783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:25.139693 25783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:25.139715 25783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:25.139736 25783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:25.139763 25783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:25.141341 25783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:46773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:25.141620 25783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:25.141855 25783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:25.146999 25793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:25.146979 25791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:25.147106 25783 server_base.cc:1034] running on GCE node
W20260808 07:11:25.146973 25789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:25.147627 25783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:25.147927 25783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:25.149101 25783 hybrid_clock.cc:648] HybridClock initialized: now 1786173085149088 us; error 33 us; skew 500 ppm
Aug 08 07:11:25 dist-test-slave-6k22 krb5kdc[25770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173085, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:25.151824 25783 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:25.152297 25783 webserver.cc:460] Webserver started at http://127.22.2.190:38959/ using document root <none> and password file <none>
I20260808 07:11:25.152494 25783 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:25.152537 25783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:25.152630 25783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:25.153684 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1"
format_stamp: "Formatted at 2026-08-08 07:11:25 on dist-test-slave-6k22"
server_key: "023e3782895d3c19691fb1d5099d51ea"
server_key_iv: "d862a99c48fe1845989775a90d7614bb"
server_key_version: "encryptionkey@0"
I20260808 07:11:25.154000 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1"
format_stamp: "Formatted at 2026-08-08 07:11:25 on dist-test-slave-6k22"
server_key: "023e3782895d3c19691fb1d5099d51ea"
server_key_iv: "d862a99c48fe1845989775a90d7614bb"
server_key_version: "encryptionkey@0"
I20260808 07:11:25.155274 25783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:11:25.156075 25801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:25.156250 25783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:25.156375 25783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1"
format_stamp: "Formatted at 2026-08-08 07:11:25 on dist-test-slave-6k22"
server_key: "023e3782895d3c19691fb1d5099d51ea"
server_key_iv: "d862a99c48fe1845989775a90d7614bb"
server_key_version: "encryptionkey@0"
I20260808 07:11:25.156456 25783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:25.170328 25783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:25.172329 25783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:25.172526 25783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:25.176874 25783 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:46773
I20260808 07:11:25.176910 25860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:46773 every 8 connection(s)
I20260808 07:11:25.177491 25783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:25.178181 25861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:25.179189 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 25783
I20260808 07:11:25.179272 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:25.179476 22538 external_mini_cluster.cc:1428] Setting key 28141da8a377163343359bff23b77bc0
I20260808 07:11:25.181198 25861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1: Bootstrap starting.
I20260808 07:11:25.181859 25861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:25.182199 25861 log.cc:824] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:25.182940 25861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1: No bootstrap required, opened a new log
I20260808 07:11:25.185242 25861 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46773 } }
I20260808 07:11:25.185417 25861 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:25.185458 25861 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1cf0743d5f74f0eae81abe1dbb8b5f1, State: Initialized, Role: FOLLOWER
I20260808 07:11:25.185616 25861 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46773 } }
I20260808 07:11:25.185719 25861 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:25.185765 25861 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:25.185820 25861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:25.186342 25861 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46773 } }
I20260808 07:11:25.186462 25861 leader_election.cc:302] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1cf0743d5f74f0eae81abe1dbb8b5f1; no voters:
I20260808 07:11:25.186625 25861 leader_election.cc:288] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:25.186657 25868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:25.186789 25868 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [term 1 LEADER]: Becoming Leader. State: Replica: e1cf0743d5f74f0eae81abe1dbb8b5f1, State: Running, Role: LEADER
I20260808 07:11:25.186913 25861 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:25.186949 25868 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46773 } }
I20260808 07:11:25.187386 25869 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46773 } } }
I20260808 07:11:25.187469 25869 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:25.187475 25870 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1cf0743d5f74f0eae81abe1dbb8b5f1. Latest consensus state: current_term: 1 leader_uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cf0743d5f74f0eae81abe1dbb8b5f1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46773 } } }
I20260808 07:11:25.187629 25870 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1 [sys.catalog]: This master's current role is: LEADER
Aug 08 07:11:25 dist-test-slave-6k22 krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:25.188668 25875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:25.189229 25875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:25.190984 25875 catalog_manager.cc:1347] Generated new cluster ID: 0af1b203878d47fc8e3fe9c69ed57b5d
I20260808 07:11:25.191045 25875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:25.192868 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:25.179993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.22.2.190:46773)
0808 07:11:25.180205 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:25.180214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:25.180248 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:11:25.180900 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:25.180909 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:25.180973 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:25.181126 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:25.181795 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.183161 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.184082 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.184776 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.184891 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:25.188481 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:25.188485 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:25.188486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:25.188500 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:25.190860 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:25.191384 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:25.191388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:25.191388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:25.191500 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:25.191837 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:25.191838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:25.191839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:25.192128 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:25.192335 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:25.192444 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:11:25.192603 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260808 07:11:25.194028 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:11:25.197697 25875 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:25.198657 25875 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:25.209561 25875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1cf0743d5f74f0eae81abe1dbb8b5f1: Generated new TSK 0
I20260808 07:11:25.209862 25875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:11:25.213280 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:25.208631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.22.2.190:46773)
0808 07:11:25.208676 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:25.208678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:25.208877 (+ 199us) server_negotiation.cc:407] Connection header received
0808 07:11:25.208920 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:25.208922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:25.208937 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:25.209015 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:25.209656 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.210406 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.211535 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.211912 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.211985 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:25.212740 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:25.212741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:25.212742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:25.212951 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:25.212971 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:25.212977 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:25.213035 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:25.213128 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 07:11:25.213157 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:11:25.215679 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 25783
2026-08-08T07:11:25Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1468 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:25 dist-test-slave-6k22 krb5kdc[25898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:25 dist-test-slave-6k22 krb5kdc[25898](info): set up 2 sockets
Aug 08 07:11:25 dist-test-slave-6k22 krb5kdc[25898](info): commencing operation
krb5kdc: starting...
W20260808 07:11:26.378567 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:26 dist-test-slave-6k22 krb5kdc[25898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:26.515031 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35773
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:26.598769 25912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:26.599025 25912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:26.599066 25912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:26.600376 25912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:26.600442 25912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:26.600464 25912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:26.600484 25912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:26.600517 25912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:26.601990 25912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:26.602278 25912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:26.602514 25912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:26.607635 25918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:26.607637 25920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:26.607770 25912 server_base.cc:1034] running on GCE node
W20260808 07:11:26.607702 25917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:26.608147 25912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:26.608513 25912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:26.609666 25912 hybrid_clock.cc:648] HybridClock initialized: now 1786173086609644 us; error 37 us; skew 500 ppm
Aug 08 07:11:26 dist-test-slave-6k22 krb5kdc[25898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173086, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:26.612206 25912 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:26.612615 25912 webserver.cc:460] Webserver started at http://127.22.2.190:46873/ using document root <none> and password file <none>
I20260808 07:11:26.612802 25912 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:26.612841 25912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:26.612932 25912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:26.613969 25912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "779518d399c04f6fbaaade4b8169bfbb"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "c8e922042ca8d3b49a0bb8fcffc829f2"
server_key_iv: "0daa574adcfdda28b7d2ec725c64fcac"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.614303 25912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "779518d399c04f6fbaaade4b8169bfbb"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "c8e922042ca8d3b49a0bb8fcffc829f2"
server_key_iv: "0daa574adcfdda28b7d2ec725c64fcac"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.615664 25912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:26.616425 25928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:26.616595 25912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:26.616708 25912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "779518d399c04f6fbaaade4b8169bfbb"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "c8e922042ca8d3b49a0bb8fcffc829f2"
server_key_iv: "0daa574adcfdda28b7d2ec725c64fcac"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.616793 25912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:26.643069 25912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:26.645356 25912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:26.645565 25912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:26.649778 25912 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35773
I20260808 07:11:26.649813 25985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35773 every 8 connection(s)
I20260808 07:11:26.650183 25912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:26.650723 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 25912
I20260808 07:11:26.650846 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:26.651047 22538 external_mini_cluster.cc:1428] Setting key e2c3082e0682f99eb02192d6d5e203d8
I20260808 07:11:26.651602 25986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:26.654983 25986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb: Bootstrap starting.
I20260808 07:11:26.655684 25986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:26.656009 25986 log.cc:824] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:26.656870 25986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb: No bootstrap required, opened a new log
Aug 08 07:11:26 dist-test-slave-6k22 krb5kdc[25898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:26.658953 25986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779518d399c04f6fbaaade4b8169bfbb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35773 } }
I20260808 07:11:26.659140 25986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:26.659212 25986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779518d399c04f6fbaaade4b8169bfbb, State: Initialized, Role: FOLLOWER
I20260808 07:11:26.659350 25986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779518d399c04f6fbaaade4b8169bfbb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35773 } }
I20260808 07:11:26.659440 25986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:26.659487 25986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:26.659536 25986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:26.660094 25986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779518d399c04f6fbaaade4b8169bfbb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35773 } }
I20260808 07:11:26.660209 25986 leader_election.cc:302] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 779518d399c04f6fbaaade4b8169bfbb; no voters:
I20260808 07:11:26.660370 25986 leader_election.cc:288] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:26.660404 25991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:26.660537 25991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [term 1 LEADER]: Becoming Leader. State: Replica: 779518d399c04f6fbaaade4b8169bfbb, State: Running, Role: LEADER
I20260808 07:11:26.660650 25986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:26.660686 25991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779518d399c04f6fbaaade4b8169bfbb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35773 } }
I20260808 07:11:26.661056 25991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 779518d399c04f6fbaaade4b8169bfbb. Latest consensus state: current_term: 1 leader_uuid: "779518d399c04f6fbaaade4b8169bfbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779518d399c04f6fbaaade4b8169bfbb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35773 } } }
I20260808 07:11:26.661212 25991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:26.661319 25992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "779518d399c04f6fbaaade4b8169bfbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779518d399c04f6fbaaade4b8169bfbb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35773 } } }
I20260808 07:11:26.661461 25992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:26.661489 26004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:26.662061 26004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:26.663017 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:26.651672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.22.2.190:35773)
0808 07:11:26.651879 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:26.651889 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:26.651922 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:11:26.652492 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:26.652501 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:26.652567 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:26.652723 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:26.653357 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.654730 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.655534 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.656199 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.656311 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.658708 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:26.658712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:26.658713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:26.658725 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:26.660878 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:26.661281 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:26.661284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:26.661285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:26.661391 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:26.661644 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:26.661645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:26.661646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:26.662007 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:26.662178 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:26.662676 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 07:11:26.662824 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260808 07:11:26.663883 26004 catalog_manager.cc:1347] Generated new cluster ID: cda6592ef41d4afab1d634d855435f73
I20260808 07:11:26.663937 26004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:26.676821 26004 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:26.677579 26004 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:26.683104 26004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 779518d399c04f6fbaaade4b8169bfbb: Generated new TSK 0
I20260808 07:11:26.683372 26004 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:26.724030 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35773
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:26.807363 26017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:26.807592 26017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:26.807618 26017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:26.808707 26017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:26.808741 26017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:26.808768 26017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:26.810210 26017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:26.810400 26017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:26.810572 26017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:26.815586 26022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:26.815591 26026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:26.815850 26017 server_base.cc:1034] running on GCE node
W20260808 07:11:26.815637 26024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:26.816128 26017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:26.816340 26017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:26.817483 26017 hybrid_clock.cc:648] HybridClock initialized: now 1786173086817461 us; error 37 us; skew 500 ppm
Aug 08 07:11:26 dist-test-slave-6k22 krb5kdc[25898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173086, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:26.820240 26017 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:26.820691 26017 webserver.cc:460] Webserver started at http://127.22.2.129:44863/ using document root <none> and password file <none>
I20260808 07:11:26.820945 26017 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:26.820993 26017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:26.821110 26017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:26.822302 26017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "6c2ac06ab53d414489f71b52363daeea"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "3765eddcc6b7dc4994d1beced71d1758"
server_key_iv: "8efaba41580cfede67ceb8d11b4588e0"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.822676 26017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "6c2ac06ab53d414489f71b52363daeea"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "3765eddcc6b7dc4994d1beced71d1758"
server_key_iv: "8efaba41580cfede67ceb8d11b4588e0"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.824060 26017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:26.824774 26034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:26.824960 26017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:26.825063 26017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "6c2ac06ab53d414489f71b52363daeea"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "3765eddcc6b7dc4994d1beced71d1758"
server_key_iv: "8efaba41580cfede67ceb8d11b4588e0"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.825145 26017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:26.838320 26017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:26.840797 26017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:26.840970 26017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:26.841270 26017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:26.841611 26017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:26.841665 26017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:26.841701 26017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:26.841747 26017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:26.847755 26017 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:35569
I20260808 07:11:26.847831 26163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:35569 every 8 connection(s)
I20260808 07:11:26.848170 26017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:26.849936 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 26017
I20260808 07:11:26.850028 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:26.850260 22538 external_mini_cluster.cc:1428] Setting key 1d4fc7f6ec9df663befb94e4fd373d72
Aug 08 07:11:26 dist-test-slave-6k22 krb5kdc[25898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173086, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:26.859432 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:26.849002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:60115 (local address 127.22.2.190:35773)
0808 07:11:26.849094 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:26.849097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:26.849225 (+ 128us) server_negotiation.cc:407] Connection header received
0808 07:11:26.850296 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:26.850298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:26.850314 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:26.850391 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:26.851301 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.851873 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.853477 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.853884 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.853956 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.856509 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:26.856510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:26.856511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:26.856519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:26.857933 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:26.858413 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:26.858414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:26.858415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:26.858478 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:26.858742 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:26.858743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:26.858743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:26.858864 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:26.858937 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:26.859225 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 07:11:26.859339 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 07:11:26.859642 26166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:26.848929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35773 (local address 127.22.2.129:60115)
0808 07:11:26.849103 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:26.849148 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:26.850176 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:26.850550 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:26.850555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:26.850899 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:26.851199 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:26.851206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.851962 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.851963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:26.853373 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:26.853376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.853983 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.853984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:26.854133 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.854658 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:26.854669 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:26.856408 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:26.858072 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:26.858080 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:26.858082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:26.858318 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:26.858550 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:26.858551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:26.858551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:26.858667 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:26.858940 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:26.858949 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:26.859091 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 07:11:26.859203 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:11:26.859366 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
I20260808 07:11:26.860162 26164 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35773
I20260808 07:11:26.860275 26164 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:26.860464 26164 heartbeater.cc:511] Master 127.22.2.190:35773 requested a full tablet report, sending...
I20260808 07:11:26.861083 25946 ts_manager.cc:194] Registered new tserver with Master: 6c2ac06ab53d414489f71b52363daeea (127.22.2.129:35569)
I20260808 07:11:26.861797 25946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:60115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:26.878487 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35773
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:26.964121 26169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:26.964399 26169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:26.964445 26169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:26.965732 26169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:26.965807 26169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:26.965854 26169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:26.967337 26169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:26.967648 26169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:26.967900 26169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:26.973034 26176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:26.973212 26175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:26.973060 26178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:26.973316 26169 server_base.cc:1034] running on GCE node
I20260808 07:11:26.973572 26169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:26.973841 26169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:26.975011 26169 hybrid_clock.cc:648] HybridClock initialized: now 1786173086975002 us; error 39 us; skew 500 ppm
Aug 08 07:11:26 dist-test-slave-6k22 krb5kdc[25898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173086, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:26.977625 26169 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:26.978266 26169 webserver.cc:460] Webserver started at http://127.22.2.130:32965/ using document root <none> and password file <none>
I20260808 07:11:26.978519 26169 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:26.978590 26169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:26.978806 26169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:26.980010 26169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "0641bfe4721f4aa0bf73ff1b0fb1e1fe"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "32bd36e9e461b6acdc381a1f88fb5140"
server_key_iv: "a567d0790635dd7fcc565be49974771f"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.980391 26169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "0641bfe4721f4aa0bf73ff1b0fb1e1fe"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "32bd36e9e461b6acdc381a1f88fb5140"
server_key_iv: "a567d0790635dd7fcc565be49974771f"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.987098 26169 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.000s
I20260808 07:11:26.988220 26185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:26.988492 26169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:26.988622 26169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "0641bfe4721f4aa0bf73ff1b0fb1e1fe"
format_stamp: "Formatted at 2026-08-08 07:11:26 on dist-test-slave-6k22"
server_key: "32bd36e9e461b6acdc381a1f88fb5140"
server_key_iv: "a567d0790635dd7fcc565be49974771f"
server_key_version: "encryptionkey@0"
I20260808 07:11:26.988719 26169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:26.998809 26169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:27.001392 26169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:27.001573 26169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:27.001842 26169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:27.002167 26169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:27.002223 26169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:27.002267 26169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:27.002297 26169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:27.009402 26169 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:45735
I20260808 07:11:27.009466 26317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:45735 every 8 connection(s)
I20260808 07:11:27.009928 26169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:27.014194 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 26169
I20260808 07:11:27.014312 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:27.014510 22538 external_mini_cluster.cc:1428] Setting key 18971cc3ce4b9c86f6123035a2d17b6a
Aug 08 07:11:27 dist-test-slave-6k22 krb5kdc[25898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173086, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:27.024489 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:27.010815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:43543 (local address 127.22.2.190:35773)
0808 07:11:27.010891 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:27.010893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:27.011105 (+ 212us) server_negotiation.cc:407] Connection header received
0808 07:11:27.011739 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:27.011740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:27.011753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:27.011803 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:27.012950 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.013482 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.014896 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.015165 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.015205 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.020682 (+ 5477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:27.020684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:27.020684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:27.020693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:27.022719 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:27.023309 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:27.023311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:27.023312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:27.023377 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:27.023709 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:27.023710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:27.023710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:27.023840 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:27.023903 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:27.024331 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 07:11:27.024418 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:11:27.024720 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:27.010783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35773 (local address 127.22.2.130:43543)
0808 07:11:27.010969 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:27.011021 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:27.011548 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:27.011926 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:27.011931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:27.012628 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:27.012841 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.012847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.013598 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.013599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.014787 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.014791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.015308 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.015309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.015456 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.018610 (+ 3154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:27.018619 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:27.020544 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:27.022829 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:27.022839 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:27.022842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:27.023214 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:27.023452 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:27.023453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:27.023453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:27.023638 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:27.023916 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:27.023925 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:27.024128 (+ 203us) client_negotiation.cc:769] Sending connection context
0808 07:11:27.024253 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:11:27.024445 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260808 07:11:27.025233 26318 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35773
I20260808 07:11:27.025352 26318 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:27.025556 26318 heartbeater.cc:511] Master 127.22.2.190:35773 requested a full tablet report, sending...
I20260808 07:11:27.025983 25946 ts_manager.cc:194] Registered new tserver with Master: 0641bfe4721f4aa0bf73ff1b0fb1e1fe (127.22.2.130:45735)
I20260808 07:11:27.026381 25946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:43543
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:27.046595 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35773
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:27.130977 26323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:27.131206 26323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:27.131228 26323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:27.132447 26323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:27.132493 26323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:27.132519 26323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:27.133895 26323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:27.134078 26323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:27.134251 26323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:27.139488 26330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:27.139418 26332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:27.139410 26329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:27.139866 26323 server_base.cc:1034] running on GCE node
I20260808 07:11:27.140089 26323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:27.140339 26323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:27.141481 26323 hybrid_clock.cc:648] HybridClock initialized: now 1786173087141462 us; error 30 us; skew 500 ppm
Aug 08 07:11:27 dist-test-slave-6k22 krb5kdc[25898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173087, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:27.144292 26323 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:27.144809 26323 webserver.cc:460] Webserver started at http://127.22.2.131:35409/ using document root <none> and password file <none>
I20260808 07:11:27.145047 26323 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:27.145092 26323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:27.145246 26323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:27.146317 26323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "88e3508b791e430c8b822262cbef7c6e"
format_stamp: "Formatted at 2026-08-08 07:11:27 on dist-test-slave-6k22"
server_key: "f8732f584dd7d3ca72d190054c0edf07"
server_key_iv: "326cc74aa65ebf83cc8d9937d3546614"
server_key_version: "encryptionkey@0"
I20260808 07:11:27.146679 26323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "88e3508b791e430c8b822262cbef7c6e"
format_stamp: "Formatted at 2026-08-08 07:11:27 on dist-test-slave-6k22"
server_key: "f8732f584dd7d3ca72d190054c0edf07"
server_key_iv: "326cc74aa65ebf83cc8d9937d3546614"
server_key_version: "encryptionkey@0"
I20260808 07:11:27.148087 26323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:27.148802 26342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:27.148981 26323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:27.149071 26323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "88e3508b791e430c8b822262cbef7c6e"
format_stamp: "Formatted at 2026-08-08 07:11:27 on dist-test-slave-6k22"
server_key: "f8732f584dd7d3ca72d190054c0edf07"
server_key_iv: "326cc74aa65ebf83cc8d9937d3546614"
server_key_version: "encryptionkey@0"
I20260808 07:11:27.149158 26323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:27.176139 26323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:27.178007 26323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:27.178179 26323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:27.178470 26323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:27.178791 26323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:27.178844 26323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:27.178887 26323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:27.178917 26323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:27.185117 26323 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:40849
I20260808 07:11:27.185159 26465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:40849 every 8 connection(s)
I20260808 07:11:27.185539 26323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:11:27 dist-test-slave-6k22 krb5kdc[25898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173087, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:27.192682 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 26323
I20260808 07:11:27.192824 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:27.193050 22538 external_mini_cluster.cc:1428] Setting key d259057267fdf9e058fbba2f6624f52d
I20260808 07:11:27.195083 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:27.186277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:53437 (local address 127.22.2.190:35773)
0808 07:11:27.186353 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:27.186355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:27.186575 (+ 220us) server_negotiation.cc:407] Connection header received
0808 07:11:27.187042 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:27.187043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:27.187054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:27.187105 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:27.188114 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.188605 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.189607 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.189794 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.189852 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.192180 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:27.192181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:27.192181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:27.192189 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:27.193918 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:27.194303 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:27.194304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:27.194304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:27.194360 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:27.194540 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:27.194540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:27.194540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:27.194684 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:27.194760 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:27.194956 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 07:11:27.195017 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:11:27.195271 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:27.186272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35773 (local address 127.22.2.131:53437)
0808 07:11:27.186450 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:27.186486 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:27.186961 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:27.187179 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:27.187183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:27.187836 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:27.188034 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.188039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.188697 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.188698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.189530 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.189533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.189872 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.189873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.189970 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.190515 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:27.190527 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:27.192095 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:27.193986 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:27.193994 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:27.193997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:27.194237 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:27.194406 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:27.194407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:27.194407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:27.194495 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:27.194750 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:27.194753 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:27.194861 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 07:11:27.194941 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:11:27.195061 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
I20260808 07:11:27.195765 26467 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35773
I20260808 07:11:27.195852 26467 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:27.195997 26467 heartbeater.cc:511] Master 127.22.2.190:35773 requested a full tablet report, sending...
I20260808 07:11:27.196326 25946 ts_manager.cc:194] Registered new tserver with Master: 88e3508b791e430c8b822262cbef7c6e (127.22.2.131:40849)
I20260808 07:11:27.196700 25946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:53437
I20260808 07:11:27.204938 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:11:27 dist-test-slave-6k22 krb5kdc[25898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 07:11:27 dist-test-slave-6k22 krb5kdc[25898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:27.229051 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:27.221435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54802 (local address 127.22.2.190:35773)
0808 07:11:27.221501 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:27.221504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:27.221618 (+ 114us) server_negotiation.cc:407] Connection header received
0808 07:11:27.221724 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:27.221725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:27.221737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:27.221822 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:27.222420 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.222928 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.223938 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.224124 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.224182 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.226423 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:27.226425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:27.226426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:27.226433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:27.227805 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:27.228158 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:27.228160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:27.228160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:27.228219 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:27.228459 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:27.228460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:27.228460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:27.228654 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:27.228756 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:27.228861 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 07:11:27.228943 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 07:11:27.229321 25946 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54802
I20260808 07:11:27.229763 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 26017
I20260808 07:11:27.237386 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 26169
I20260808 07:11:27.242971 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 26323
I20260808 07:11:27.248652 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 25912
2026-08-08T07:11:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2032 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:27 dist-test-slave-6k22 krb5kdc[26482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:27 dist-test-slave-6k22 krb5kdc[26482](info): set up 2 sockets
Aug 08 07:11:27 dist-test-slave-6k22 krb5kdc[26482](info): commencing operation
krb5kdc: starting...
W20260808 07:11:28.347543 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:28 dist-test-slave-6k22 krb5kdc[26482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:28.472697 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37741
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:37741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:28.558393 26495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:28.558624 26495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:28.558647 26495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:28.559937 26495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:28.559984 26495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:28.560006 26495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:28.560040 26495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:28.560071 26495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:28.561686 26495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:37741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:28.561969 26495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:28.562222 26495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:28.567324 26503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:28.567324 26500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:28.567340 26501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:28.567464 26495 server_base.cc:1034] running on GCE node
I20260808 07:11:28.567893 26495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:28.568153 26495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:28.569324 26495 hybrid_clock.cc:648] HybridClock initialized: now 1786173088569318 us; error 37 us; skew 500 ppm
Aug 08 07:11:28 dist-test-slave-6k22 krb5kdc[26482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173088, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:28.574900 26495 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:28.575417 26495 webserver.cc:460] Webserver started at http://127.22.2.190:41571/ using document root <none> and password file <none>
I20260808 07:11:28.575717 26495 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:28.575789 26495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:28.575956 26495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:28.577049 26495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "229f245c5f974e2f8e692dfdec7acea5"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "46d6f1085025de6f968852bc5f42f6ec"
server_key_iv: "92ef83438db87043a07ef71f7f7ff3ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.577433 26495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "229f245c5f974e2f8e692dfdec7acea5"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "46d6f1085025de6f968852bc5f42f6ec"
server_key_iv: "92ef83438db87043a07ef71f7f7ff3ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.578965 26495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:28.579809 26510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:28.579986 26495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:28.580096 26495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "229f245c5f974e2f8e692dfdec7acea5"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "46d6f1085025de6f968852bc5f42f6ec"
server_key_iv: "92ef83438db87043a07ef71f7f7ff3ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.580171 26495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:28.604503 26495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:28.606390 26495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:28.606562 26495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:28.610980 26495 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:37741
I20260808 07:11:28.611021 26572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:37741 every 8 connection(s)
I20260808 07:11:28.611585 26495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:28.612275 26573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:28.614624 26573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5: Bootstrap starting.
I20260808 07:11:28.615280 26573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:28.615612 26573 log.cc:824] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:28.616372 26573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5: No bootstrap required, opened a new log
I20260808 07:11:28.617990 26573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229f245c5f974e2f8e692dfdec7acea5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37741 } }
I20260808 07:11:28.618129 26573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:28.618153 26573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229f245c5f974e2f8e692dfdec7acea5, State: Initialized, Role: FOLLOWER
I20260808 07:11:28.618335 26573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229f245c5f974e2f8e692dfdec7acea5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37741 } }
I20260808 07:11:28.618414 26573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:28.618436 26573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:28.618507 26573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:28.618662 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 26495
I20260808 07:11:28.618782 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:28.618969 22538 external_mini_cluster.cc:1428] Setting key 6cfcdb227a0ff445bca278967568dcc6
I20260808 07:11:28.619704 26573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229f245c5f974e2f8e692dfdec7acea5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37741 } }
I20260808 07:11:28.619853 26573 leader_election.cc:302] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 229f245c5f974e2f8e692dfdec7acea5; no voters:
I20260808 07:11:28.620039 26573 leader_election.cc:288] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:28.620167 26579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:28.620314 26573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:28.620350 26579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [term 1 LEADER]: Becoming Leader. State: Replica: 229f245c5f974e2f8e692dfdec7acea5, State: Running, Role: LEADER
I20260808 07:11:28.620568 26579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229f245c5f974e2f8e692dfdec7acea5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37741 } }
I20260808 07:11:28.620977 26581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "229f245c5f974e2f8e692dfdec7acea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229f245c5f974e2f8e692dfdec7acea5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37741 } } }
I20260808 07:11:28.621086 26581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:28.621232 26581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 229f245c5f974e2f8e692dfdec7acea5. Latest consensus state: current_term: 1 leader_uuid: "229f245c5f974e2f8e692dfdec7acea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229f245c5f974e2f8e692dfdec7acea5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37741 } } }
I20260808 07:11:28.621289 26581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:28.621543 26588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:28.622206 26588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:28.623908 26588 catalog_manager.cc:1347] Generated new cluster ID: 6598c1ba0de34032b9ae0d9f5db84b8b
I20260808 07:11:28.623971 26588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:11:28 dist-test-slave-6k22 krb5kdc[26482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:28.631507 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:28.619488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.22.2.190:37741)
0808 07:11:28.619711 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:28.619718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:28.619744 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:11:28.619870 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:28.619878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:28.619935 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:28.620111 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:28.620943 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.622437 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.623377 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.624007 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.624126 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.627138 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:28.627142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:28.627142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:28.627155 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:28.629822 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:28.630242 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:28.630245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:28.630245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:28.630354 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:28.630576 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:28.630577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:28.630577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:28.630881 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:28.631039 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:28.631161 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 07:11:28.631293 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":135,"threads_started":1}
I20260808 07:11:28.638645 26588 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:28.639367 26588 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:28.650660 26588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 229f245c5f974e2f8e692dfdec7acea5: Generated new TSK 0
I20260808 07:11:28.650902 26588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:28.682706 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37741
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:28.771070 26601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:28.771292 26601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:28.771317 26601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:28.772526 26601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:28.772572 26601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:28.772599 26601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:28.774060 26601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:28.774255 26601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:28.774447 26601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:28.779520 26608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:28.779582 26607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:28.779618 26610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:28.780018 26601 server_base.cc:1034] running on GCE node
I20260808 07:11:28.780260 26601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:28.780504 26601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:28.781702 26601 hybrid_clock.cc:648] HybridClock initialized: now 1786173088781689 us; error 36 us; skew 500 ppm
Aug 08 07:11:28 dist-test-slave-6k22 krb5kdc[26482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173088, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:28.784500 26601 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:28.785005 26601 webserver.cc:460] Webserver started at http://127.22.2.129:44811/ using document root <none> and password file <none>
I20260808 07:11:28.785199 26601 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:28.785240 26601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:28.785333 26601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:28.786336 26601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "3638c60405f5470cae7b85308426b9ac"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "94fe09a732c7327ad39a3d889a8526a7"
server_key_iv: "b7c6054ed5e0726108ae956909d7c7e3"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.786643 26601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "3638c60405f5470cae7b85308426b9ac"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "94fe09a732c7327ad39a3d889a8526a7"
server_key_iv: "b7c6054ed5e0726108ae956909d7c7e3"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.788000 26601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:28.788719 26619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:28.788892 26601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:28.788978 26601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "3638c60405f5470cae7b85308426b9ac"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "94fe09a732c7327ad39a3d889a8526a7"
server_key_iv: "b7c6054ed5e0726108ae956909d7c7e3"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.789036 26601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:28.805294 26601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:28.807174 26601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:28.807289 26601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:28.807490 26601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:28.807839 26601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:28.807875 26601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:28.807921 26601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:28.807947 26601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:28.813781 26601 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:44787
I20260808 07:11:28.813868 26744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:44787 every 8 connection(s)
I20260808 07:11:28.814198 26601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:28.818851 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 26601
I20260808 07:11:28.818943 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:28.819129 22538 external_mini_cluster.cc:1428] Setting key bed4238d18ed1850f9b017a2b0af0c8d
Aug 08 07:11:28 dist-test-slave-6k22 krb5kdc[26482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173088, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:28.824210 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:28.814988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:39711 (local address 127.22.2.190:37741)
0808 07:11:28.815057 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:28.815060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:28.815288 (+ 228us) server_negotiation.cc:407] Connection header received
0808 07:11:28.816304 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:28.816305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:28.816317 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:28.816379 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:28.816962 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.817460 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.818445 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.818684 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.818752 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.821195 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:28.821197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:28.821197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:28.821207 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:28.822734 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:28.823175 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:28.823176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:28.823177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:28.823236 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:28.823510 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:28.823511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:28.823511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:28.823662 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:28.823743 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:28.824024 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 07:11:28.824120 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:11:28.824453 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:28.814965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37741 (local address 127.22.2.129:39711)
0808 07:11:28.815149 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:28.815194 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:28.816207 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:28.816434 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:28.816437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:28.816693 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:28.816891 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.816894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.817540 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.817540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.818368 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.818370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.818763 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.818764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.818873 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.819491 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:28.819504 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:28.821101 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:28.822855 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:28.822863 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:28.822865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:28.823096 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:28.823320 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:28.823321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:28.823321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:28.823433 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:28.823733 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:28.823740 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:28.823886 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 07:11:28.824001 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:11:28.824164 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260808 07:11:28.825009 26745 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37741
I20260808 07:11:28.825124 26745 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:28.825316 26745 heartbeater.cc:511] Master 127.22.2.190:37741 requested a full tablet report, sending...
I20260808 07:11:28.825942 26531 ts_manager.cc:194] Registered new tserver with Master: 3638c60405f5470cae7b85308426b9ac (127.22.2.129:44787)
I20260808 07:11:28.826927 26531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:39711
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:28.846807 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37741
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:28.931609 26751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:28.931895 26751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:28.931924 26751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:28.933054 26751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:28.933090 26751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:28.933120 26751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:28.934576 26751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:28.934768 26751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:28.934954 26751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:28.940075 26759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:28.940073 26761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:28.940073 26758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:28.940231 26751 server_base.cc:1034] running on GCE node
I20260808 07:11:28.940490 26751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:28.940690 26751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:28.941843 26751 hybrid_clock.cc:648] HybridClock initialized: now 1786173088941831 us; error 26 us; skew 500 ppm
Aug 08 07:11:28 dist-test-slave-6k22 krb5kdc[26482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173088, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:28.944407 26751 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:28.944846 26751 webserver.cc:460] Webserver started at http://127.22.2.130:36701/ using document root <none> and password file <none>
I20260808 07:11:28.945096 26751 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:28.945142 26751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:28.945322 26751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:28.946398 26751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "c3cc5ef59a4c40f490b50f825d71ad6d"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "5d0b4ecd43f3c9af293714dfa7a66741"
server_key_iv: "7c74541272b5da91aafcd3e530e1eaea"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.946758 26751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "c3cc5ef59a4c40f490b50f825d71ad6d"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "5d0b4ecd43f3c9af293714dfa7a66741"
server_key_iv: "7c74541272b5da91aafcd3e530e1eaea"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.948176 26751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:11:28.948915 26768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:28.949107 26751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:28.949201 26751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "c3cc5ef59a4c40f490b50f825d71ad6d"
format_stamp: "Formatted at 2026-08-08 07:11:28 on dist-test-slave-6k22"
server_key: "5d0b4ecd43f3c9af293714dfa7a66741"
server_key_iv: "7c74541272b5da91aafcd3e530e1eaea"
server_key_version: "encryptionkey@0"
I20260808 07:11:28.949272 26751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:28.966728 26751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:28.968878 26751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:28.969030 26751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:28.969300 26751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:28.969631 26751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:28.969686 26751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:28.969727 26751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:28.969756 26751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:28.975711 26751 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41743
I20260808 07:11:28.975760 26894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41743 every 8 connection(s)
I20260808 07:11:28.976121 26751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:28.982637 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 26751
I20260808 07:11:28.982770 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
Aug 08 07:11:28 dist-test-slave-6k22 krb5kdc[26482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173088, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:28.983000 22538 external_mini_cluster.cc:1428] Setting key 772164e769d9e385031d3ef58d8c4d6b
I20260808 07:11:28.986678 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:28.976911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:38087 (local address 127.22.2.190:37741)
0808 07:11:28.976965 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:28.976968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:28.977228 (+ 260us) server_negotiation.cc:407] Connection header received
0808 07:11:28.978221 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:28.978223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:28.978237 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:28.978280 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:28.979212 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.979905 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.980882 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.981070 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.981122 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.983469 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:28.983471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:28.983471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:28.983478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:28.985036 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:28.985591 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:28.985592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:28.985593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:28.985670 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:28.985948 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:28.985948 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:28.985949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:28.986113 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:28.986188 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:28.986464 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 07:11:28.986559 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:28.986701 26897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:28.976916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37741 (local address 127.22.2.130:38087)
0808 07:11:28.977098 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:28.977135 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:28.978099 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:28.978371 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:28.978375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:28.978781 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:28.979093 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.979100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.979993 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.979994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.980806 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.980809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.981145 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.981146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.981265 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.981740 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:28.981750 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:28.983380 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:28.985121 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:28.985128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:28.985130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:28.985492 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:28.985758 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:28.985759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:28.985759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:28.985878 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:28.986180 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:28.986185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:28.986332 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 07:11:28.986418 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:11:28.986539 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260808 07:11:28.987254 26895 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37741
I20260808 07:11:28.987370 26895 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:28.987602 26895 heartbeater.cc:511] Master 127.22.2.190:37741 requested a full tablet report, sending...
I20260808 07:11:28.987983 26531 ts_manager.cc:194] Registered new tserver with Master: c3cc5ef59a4c40f490b50f825d71ad6d (127.22.2.130:41743)
I20260808 07:11:28.988371 26531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:38087
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:29.011058 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37741
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:29.097244 26900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:29.097453 26900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:29.097477 26900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:29.098567 26900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:29.098603 26900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:29.098629 26900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:29.100174 26900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:37741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:29.100356 26900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:29.100529 26900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:29.105512 26905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:29.105517 26909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:29.105522 26907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:29.105577 26900 server_base.cc:1034] running on GCE node
I20260808 07:11:29.106000 26900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:29.106241 26900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:29.107443 26900 hybrid_clock.cc:648] HybridClock initialized: now 1786173089107425 us; error 46 us; skew 500 ppm
Aug 08 07:11:29 dist-test-slave-6k22 krb5kdc[26482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173089, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:29.109982 26900 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:29.110411 26900 webserver.cc:460] Webserver started at http://127.22.2.131:35591/ using document root <none> and password file <none>
I20260808 07:11:29.110641 26900 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:29.110702 26900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:29.110885 26900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:29.112025 26900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "a700bf2eebcf463980759e0a5fba963d"
format_stamp: "Formatted at 2026-08-08 07:11:29 on dist-test-slave-6k22"
server_key: "32e8adb41ec4b2d6f34b9aeb61e33418"
server_key_iv: "4cfe0e5956cb3ac22ab8f052c613bd82"
server_key_version: "encryptionkey@0"
I20260808 07:11:29.112391 26900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "a700bf2eebcf463980759e0a5fba963d"
format_stamp: "Formatted at 2026-08-08 07:11:29 on dist-test-slave-6k22"
server_key: "32e8adb41ec4b2d6f34b9aeb61e33418"
server_key_iv: "4cfe0e5956cb3ac22ab8f052c613bd82"
server_key_version: "encryptionkey@0"
I20260808 07:11:29.113695 26900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:11:29.114441 26917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:29.114645 26900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:29.114758 26900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "a700bf2eebcf463980759e0a5fba963d"
format_stamp: "Formatted at 2026-08-08 07:11:29 on dist-test-slave-6k22"
server_key: "32e8adb41ec4b2d6f34b9aeb61e33418"
server_key_iv: "4cfe0e5956cb3ac22ab8f052c613bd82"
server_key_version: "encryptionkey@0"
I20260808 07:11:29.114830 26900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:29.131058 26900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:29.133129 26900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:29.133293 26900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:29.133543 26900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:29.133865 26900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:29.133899 26900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:29.133945 26900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:29.133972 26900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:29.139900 26900 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:40059
I20260808 07:11:29.139940 27048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:40059 every 8 connection(s)
I20260808 07:11:29.140285 26900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:11:29 dist-test-slave-6k22 krb5kdc[26482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173089, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:29.146739 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 26900
I20260808 07:11:29.146864 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:29.147074 22538 external_mini_cluster.cc:1428] Setting key 18c2879e34ee98fcd961b0c14bc91e32
I20260808 07:11:29.149935 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:29.141038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:36107 (local address 127.22.2.190:37741)
0808 07:11:29.141104 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:29.141107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:29.141317 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:11:29.142225 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:29.142226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:29.142237 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:29.142300 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:29.142878 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.143370 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.144370 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.144551 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.144606 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.146841 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:29.146843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:29.146844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:29.146852 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:29.148536 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:29.148956 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:29.148957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:29.148958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:29.149016 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:29.149280 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:29.149281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:29.149282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:29.149402 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:29.149479 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:29.149748 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 07:11:29.149843 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:11:29.150120 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:29.141011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:37741 (local address 127.22.2.131:36107)
0808 07:11:29.141183 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:29.141226 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:29.142131 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:29.142366 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:29.142369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:29.142616 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:29.142800 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:29.142804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.143450 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.143451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:29.144290 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:29.144293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.144625 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.144626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:29.144729 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.145255 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:29.145263 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:29.146715 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:29.148639 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:29.148647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:29.148649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:29.148879 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:29.149093 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:29.149094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:29.149094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:29.149210 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:29.149470 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:29.149477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:29.149615 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 07:11:29.149724 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:11:29.149885 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260808 07:11:29.150615 27049 heartbeater.cc:347] Connected to a master server at 127.22.2.190:37741
I20260808 07:11:29.150714 27049 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:29.150898 27049 heartbeater.cc:511] Master 127.22.2.190:37741 requested a full tablet report, sending...
I20260808 07:11:29.151301 26531 ts_manager.cc:194] Registered new tserver with Master: a700bf2eebcf463980759e0a5fba963d (127.22.2.131:40059)
I20260808 07:11:29.151749 26531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:36107
I20260808 07:11:29.159067 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:11:29 dist-test-slave-6k22 krb5kdc[26482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:11:29 dist-test-slave-6k22 krb5kdc[26482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:29.182830 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:29.175251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.22.2.129:44787)
0808 07:11:29.175409 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:29.175415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:29.175433 (+ 18us) server_negotiation.cc:407] Connection header received
0808 07:11:29.175462 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:29.175464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:29.175479 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:29.175545 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:29.176197 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.176932 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.177690 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.178032 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.178118 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.180397 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:29.180405 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:29.180405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:29.180412 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:29.181802 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:29.182111 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:29.182114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:29.182114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:29.182183 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:29.182337 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:29.182337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:29.182337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:29.182463 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:29.182561 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:29.182630 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:11:29.182734 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
Aug 08 07:11:29 dist-test-slave-6k22 krb5kdc[26482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:29.190716 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:29.184244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.22.2.190:37741)
0808 07:11:29.184288 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:29.184289 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:29.184316 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:11:29.184371 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:29.184371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:29.184380 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:29.184424 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:29.184828 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.185290 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.185975 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.186153 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.186201 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.188273 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:29.188275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:29.188275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:29.188282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:29.189778 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:29.190059 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:29.190060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:29.190060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:29.190111 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:29.190255 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:29.190256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:29.190256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:29.190392 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:29.190460 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:29.190539 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:11:29.190597 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:11:29.190991 26531 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50170
I20260808 07:11:29.191366 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 26601
I20260808 07:11:29.199199 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 26751
I20260808 07:11:29.209025 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 26900
I20260808 07:11:29.218024 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 26495
2026-08-08T07:11:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1968 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:29 dist-test-slave-6k22 krb5kdc[27062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:29 dist-test-slave-6k22 krb5kdc[27062](info): set up 2 sockets
Aug 08 07:11:29 dist-test-slave-6k22 krb5kdc[27062](info): commencing operation
krb5kdc: starting...
W20260808 07:11:30.311206 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:30 dist-test-slave-6k22 krb5kdc[27062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:30.438655 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35429
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:30.520298 27078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:30.520570 27078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:30.520615 27078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:30.521795 27078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:30.521867 27078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:30.521891 27078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:30.521914 27078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:30.521942 27078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:30.523451 27078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:30.523775 27078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:30.524015 27078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:30.528909 27084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:30.528914 27087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:30.528947 27083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:30.529222 27078 server_base.cc:1034] running on GCE node
I20260808 07:11:30.529448 27078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:30.529765 27078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:30.530970 27078 hybrid_clock.cc:648] HybridClock initialized: now 1786173090530927 us; error 28 us; skew 500 ppm
Aug 08 07:11:30 dist-test-slave-6k22 krb5kdc[27062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173090, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:30.533751 27078 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:30.534132 27078 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:30.534191 27078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:30.534332 27078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:30.535501 27078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "5c204cb5513744489d6204b8c6216fd1"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "f8edd4a63c09431da61c2a8160a23614"
server_key_iv: "f687f18a03e6bb1d3ee30e6e5cfbf0fe"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.535975 27078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "5c204cb5513744489d6204b8c6216fd1"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "f8edd4a63c09431da61c2a8160a23614"
server_key_iv: "f687f18a03e6bb1d3ee30e6e5cfbf0fe"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.537385 27078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:30.538143 27094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:30.538332 27078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:30.538465 27078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "5c204cb5513744489d6204b8c6216fd1"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "f8edd4a63c09431da61c2a8160a23614"
server_key_iv: "f687f18a03e6bb1d3ee30e6e5cfbf0fe"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.538546 27078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:30.552937 27078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:30.554955 27078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:30.555148 27078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:30.559748 27078 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35429
I20260808 07:11:30.559762 27151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35429 every 8 connection(s)
I20260808 07:11:30.560105 27078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:30.561177 27152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:30.563602 27152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1: Bootstrap starting.
I20260808 07:11:30.564036 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 27078
I20260808 07:11:30.564204 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:30.564296 27152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:30.564437 22538 external_mini_cluster.cc:1428] Setting key d2c7fe8c162369378c3600ab4a881c3e
I20260808 07:11:30.564684 27152 log.cc:824] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:30.565536 27152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1: No bootstrap required, opened a new log
I20260808 07:11:30.567859 27152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c204cb5513744489d6204b8c6216fd1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35429 } }
I20260808 07:11:30.568019 27152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:30.568050 27152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c204cb5513744489d6204b8c6216fd1, State: Initialized, Role: FOLLOWER
I20260808 07:11:30.568172 27152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c204cb5513744489d6204b8c6216fd1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35429 } }
I20260808 07:11:30.568262 27152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:30.568315 27152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:30.568365 27152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:30.568897 27152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c204cb5513744489d6204b8c6216fd1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35429 } }
I20260808 07:11:30.569017 27152 leader_election.cc:302] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c204cb5513744489d6204b8c6216fd1; no voters:
I20260808 07:11:30.569205 27152 leader_election.cc:288] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:30.569236 27157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:30.569362 27157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [term 1 LEADER]: Becoming Leader. State: Replica: 5c204cb5513744489d6204b8c6216fd1, State: Running, Role: LEADER
I20260808 07:11:30.569446 27157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c204cb5513744489d6204b8c6216fd1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35429 } }
I20260808 07:11:30.569483 27152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:30.569859 27158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c204cb5513744489d6204b8c6216fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c204cb5513744489d6204b8c6216fd1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35429 } } }
I20260808 07:11:30.569959 27158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:30.570214 27166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:30.570241 27159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c204cb5513744489d6204b8c6216fd1. Latest consensus state: current_term: 1 leader_uuid: "5c204cb5513744489d6204b8c6216fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c204cb5513744489d6204b8c6216fd1" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35429 } } }
I20260808 07:11:30.570423 27159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:30.571362 27166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:11:30 dist-test-slave-6k22 krb5kdc[27062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:30.573145 27166 catalog_manager.cc:1347] Generated new cluster ID: 2c8f56adb3064f83b5e5277ff505db0e
I20260808 07:11:30.573202 27166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:30.576236 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:30.565015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.22.2.190:35429)
0808 07:11:30.565202 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:30.565211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:30.565245 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:11:30.565800 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:30.565820 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:30.565881 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:30.566027 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:30.566667 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.568106 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.569046 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.569695 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.569830 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.572862 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:30.572866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:30.572866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:30.572878 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:30.574621 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:30.575016 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:30.575019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:30.575020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:30.575111 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:30.575331 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:30.575333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:30.575333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:30.575609 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:30.575804 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:30.575885 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:11:30.576022 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260808 07:11:30.583425 27166 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:30.583984 27166 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:30.597926 27166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1: Generated new TSK 0
I20260808 07:11:30.598138 27166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:30.630865 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35429
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:30.714448 27181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:30.714704 27181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:30.714749 27181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:30.715965 27181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:30.716033 27181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:30.716079 27181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:30.717600 27181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:30.717868 27181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:30.718103 27181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:30.723199 27190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:30.723238 27181 server_base.cc:1034] running on GCE node
W20260808 07:11:30.723222 27187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:30.723197 27188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:30.723698 27181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:30.723951 27181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:30.725113 27181 hybrid_clock.cc:648] HybridClock initialized: now 1786173090725102 us; error 42 us; skew 500 ppm
Aug 08 07:11:30 dist-test-slave-6k22 krb5kdc[27062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173090, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:30.727797 27181 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:30.728091 27181 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:30.728133 27181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:30.728231 27181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:30.729214 27181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "ed0f28a5b4d04b2ea58c083929894568"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "e6937cffd6ad80d6098edb362e76e71e"
server_key_iv: "48d3356599a0661fef9d99c0f2cb48f6"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.729508 27181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "ed0f28a5b4d04b2ea58c083929894568"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "e6937cffd6ad80d6098edb362e76e71e"
server_key_iv: "48d3356599a0661fef9d99c0f2cb48f6"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.730746 27181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:30.731529 27197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:30.731729 27181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:30.731817 27181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "ed0f28a5b4d04b2ea58c083929894568"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "e6937cffd6ad80d6098edb362e76e71e"
server_key_iv: "48d3356599a0661fef9d99c0f2cb48f6"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.731895 27181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:30.754993 27181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:30.757241 27181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:30.757421 27181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:30.757728 27181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:30.758105 27181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:30.758164 27181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:30.758208 27181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:30.758237 27181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:30.764214 27181 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:42715
I20260808 07:11:30.764287 27321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:42715 every 8 connection(s)
I20260808 07:11:30.764576 27181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:30.766737 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 27181
I20260808 07:11:30.766891 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:30.767112 22538 external_mini_cluster.cc:1428] Setting key ccb956d5fc87aafc23a4f11c045ccd34
Aug 08 07:11:30 dist-test-slave-6k22 krb5kdc[27062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173090, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:30.776887 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:30.765276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:58497 (local address 127.22.2.190:35429)
0808 07:11:30.765370 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:30.765372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:30.765582 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:11:30.766110 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:30.766112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:30.766122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:30.766174 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:30.768364 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.769104 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.770484 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.770817 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.770889 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.773931 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:30.773933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:30.773934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:30.773942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:30.775358 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:30.775863 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:30.775864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:30.775865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:30.775924 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:30.776192 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:30.776193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:30.776194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:30.776321 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:30.776404 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:30.776690 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 07:11:30.776781 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 07:11:30.777127 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:30.765260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35429 (local address 127.22.2.129:58497)
0808 07:11:30.765453 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:30.765489 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:30.766019 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:30.766229 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:30.766233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:30.766504 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:30.767235 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.767243 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.769221 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.769222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.770347 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.770351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.771071 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.771072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.771228 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.771953 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:30.771983 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:30.773829 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:30.775450 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:30.775458 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:30.775461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:30.775766 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:30.776003 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:30.776004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:30.776005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:30.776123 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:30.776395 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:30.776402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:30.776549 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 07:11:30.776657 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:11:30.776821 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 07:11:30.777614 27322 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35429
I20260808 07:11:30.777730 27322 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:30.777884 27322 heartbeater.cc:511] Master 127.22.2.190:35429 requested a full tablet report, sending...
I20260808 07:11:30.778379 27113 ts_manager.cc:194] Registered new tserver with Master: ed0f28a5b4d04b2ea58c083929894568 (127.22.2.129:42715)
I20260808 07:11:30.779122 27113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:58497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:30.796211 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35429
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:30.879468 27327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:30.879714 27327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:30.879739 27327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:30.880837 27327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:30.880872 27327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:30.880898 27327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:30.882328 27327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:30.882516 27327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:30.882702 27327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:30.887725 27336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:30.887866 27333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:30.887753 27334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:30.887969 27327 server_base.cc:1034] running on GCE node
I20260808 07:11:30.888198 27327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:30.888458 27327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:30.889631 27327 hybrid_clock.cc:648] HybridClock initialized: now 1786173090889623 us; error 39 us; skew 500 ppm
Aug 08 07:11:30 dist-test-slave-6k22 krb5kdc[27062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173090, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:30.892179 27327 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:30.892534 27327 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:30.892608 27327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:30.892765 27327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:30.893877 27327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "47275513ef5d4119aec689383bc96b90"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "42474bf5c05eac173de2bf4d4b640184"
server_key_iv: "7c53d1a2ee8b3d018be20e158f96d1c5"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.894266 27327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "47275513ef5d4119aec689383bc96b90"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "42474bf5c05eac173de2bf4d4b640184"
server_key_iv: "7c53d1a2ee8b3d018be20e158f96d1c5"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.895635 27327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 07:11:30.896386 27342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:30.896565 27327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:30.896669 27327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "47275513ef5d4119aec689383bc96b90"
format_stamp: "Formatted at 2026-08-08 07:11:30 on dist-test-slave-6k22"
server_key: "42474bf5c05eac173de2bf4d4b640184"
server_key_iv: "7c53d1a2ee8b3d018be20e158f96d1c5"
server_key_version: "encryptionkey@0"
I20260808 07:11:30.896754 27327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:30.908336 27327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:30.910342 27327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:30.910526 27327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:30.910797 27327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:30.911134 27327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:30.911206 27327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:30.911242 27327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:30.911291 27327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:30.917585 27327 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:36665
I20260808 07:11:30.917622 27471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:36665 every 8 connection(s)
I20260808 07:11:30.917903 27327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:30.922019 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 27327
I20260808 07:11:30.922139 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:30.922345 22538 external_mini_cluster.cc:1428] Setting key 686d61dfea74863d17c89567614e2bae
Aug 08 07:11:30 dist-test-slave-6k22 krb5kdc[27062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173090, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:30.928622 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:30.918657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:57187 (local address 127.22.2.190:35429)
0808 07:11:30.918725 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:30.918727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:30.918909 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:11:30.919484 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:30.919485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:30.919500 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:30.919587 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:30.920543 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.921048 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.922205 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.922516 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.922566 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.925248 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:30.925250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:30.925251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:30.925260 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:30.927063 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:30.927509 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:30.927511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:30.927511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:30.927617 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:30.927909 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:30.927911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:30.927911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:30.928050 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:30.928132 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:30.928432 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 07:11:30.928526 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:11:30.928921 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:30.918628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35429 (local address 127.22.2.130:57187)
0808 07:11:30.918774 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:30.918823 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:30.919343 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:30.919648 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:30.919652 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:30.919922 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:30.920430 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.920435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.921150 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.921151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.922089 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.922093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.922685 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.922686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.922814 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.923409 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:30.923420 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:30.925148 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:30.927174 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:30.927181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:30.927183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:30.927420 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:30.927694 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:30.927695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:30.927695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:30.927822 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:30.928136 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:30.928144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:30.928288 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 07:11:30.928407 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:11:30.928578 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260808 07:11:30.929400 27472 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35429
I20260808 07:11:30.929497 27472 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:30.929680 27472 heartbeater.cc:511] Master 127.22.2.190:35429 requested a full tablet report, sending...
I20260808 07:11:30.930081 27113 ts_manager.cc:194] Registered new tserver with Master: 47275513ef5d4119aec689383bc96b90 (127.22.2.130:36665)
I20260808 07:11:30.930439 27113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:57187
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:30.951030 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35429
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:31.033828 27477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:31.034088 27477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:31.034116 27477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:31.035392 27477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:31.035434 27477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:31.035462 27477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:31.037063 27477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:31.037317 27477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:31.037544 27477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:31.042799 27485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:31.042835 27477 server_base.cc:1034] running on GCE node
W20260808 07:11:31.042928 27482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:31.042799 27483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:31.043187 27477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:31.043438 27477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:31.044603 27477 hybrid_clock.cc:648] HybridClock initialized: now 1786173091044583 us; error 52 us; skew 500 ppm
Aug 08 07:11:31 dist-test-slave-6k22 krb5kdc[27062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173091, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:31.047199 27477 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:31.047632 27477 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:31.047714 27477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:31.047860 27477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:31.048904 27477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "a4746246f79f47f4835eb4ad109b9a6b"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "b3109f2c687abd1bb602564a72b0f050"
server_key_iv: "b563483370bd49e7bf3a6f4a8fd05922"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.049275 27477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "a4746246f79f47f4835eb4ad109b9a6b"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "b3109f2c687abd1bb602564a72b0f050"
server_key_iv: "b563483370bd49e7bf3a6f4a8fd05922"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.050598 27477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:31.051388 27492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.051635 27477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:31.051733 27477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "a4746246f79f47f4835eb4ad109b9a6b"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "b3109f2c687abd1bb602564a72b0f050"
server_key_iv: "b563483370bd49e7bf3a6f4a8fd05922"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.051828 27477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:31.077872 27477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:31.080581 27477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:31.080757 27477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:31.081007 27477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:31.081355 27477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:31.081416 27477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.081476 27477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:31.081513 27477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.088397 27477 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:43429
I20260808 07:11:31.088435 27619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:43429 every 8 connection(s)
I20260808 07:11:31.088712 27477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:11:31 dist-test-slave-6k22 krb5kdc[27062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173091, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:31.097004 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 27477
I20260808 07:11:31.097090 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:31.097323 22538 external_mini_cluster.cc:1428] Setting key 993ab506425097319c287c60589ada7a
I20260808 07:11:31.098585 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.089433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:52929 (local address 127.22.2.190:35429)
0808 07:11:31.089497 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.089499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.089705 (+ 206us) server_negotiation.cc:407] Connection header received
0808 07:11:31.090211 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.090212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.090222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.090285 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.091286 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.091848 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.092869 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.093062 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.093129 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.095215 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.095216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.095216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:31.095223 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.096770 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:31.097209 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:31.097211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:31.097211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:31.097269 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:31.097726 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:31.097727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:31.097728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:31.097939 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.098023 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.098402 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.098488 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:11:31.098713 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.089364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35429 (local address 127.22.2.131:52929)
0808 07:11:31.089557 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.089599 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.090129 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.090347 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.090350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.090604 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.091182 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.091187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.091947 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.091948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.092769 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.092771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.093144 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.093144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.093244 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.093739 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:31.093747 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.095115 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:31.096859 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:31.096865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:31.096866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:31.097108 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:31.097363 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:31.097364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:31.097364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:31.097496 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:31.098024 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:31.098031 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:31.098185 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.098303 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.098492 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":121,"threads_started":1}
I20260808 07:11:31.099182 27620 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35429
I20260808 07:11:31.099320 27620 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:31.099500 27620 heartbeater.cc:511] Master 127.22.2.190:35429 requested a full tablet report, sending...
I20260808 07:11:31.099920 27113 ts_manager.cc:194] Registered new tserver with Master: a4746246f79f47f4835eb4ad109b9a6b (127.22.2.131:43429)
I20260808 07:11:31.100304 27113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:52929
I20260808 07:11:31.109460 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:11:31.117955 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.110886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.22.2.190:35429)
0808 07:11:31.110977 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.110979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.111075 (+ 96us) server_negotiation.cc:407] Connection header received
0808 07:11:31.111198 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.111199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.111211 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.111273 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.111987 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.112519 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.113332 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.113569 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.113669 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.115013 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.115015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.115015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:31.115024 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.116565 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:31.117020 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:31.117021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:31.117022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:31.117089 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:31.117311 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:31.117312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:31.117313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:31.117486 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.117607 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.117752 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.117840 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 07:11:31.119656 27113 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:31.121258 27113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:31.128119 27634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.123737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:36665 (local address 127.0.0.1:39276)
0808 07:11:31.124389 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.124398 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.124442 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.124759 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.124760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.124762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:31.124818 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.124819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.126018 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.126019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.127118 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.127121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.127862 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.127863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.127925 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.127931 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.127991 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.128035 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":82,"threads_started":1}
I20260808 07:11:31.128137 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.123997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.22.2.130:36665)
0808 07:11:31.124178 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.124182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.124499 (+ 317us) server_negotiation.cc:407] Connection header received
0808 07:11:31.124546 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.124549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.124565 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.124637 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:31.124925 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.125889 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.127214 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.127776 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.127860 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.127919 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.128006 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.128046 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
I20260808 07:11:31.128239 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.123934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.22.2.129:42715)
0808 07:11:31.124134 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.124139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.124256 (+ 117us) server_negotiation.cc:407] Connection header received
0808 07:11:31.124384 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.124387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.124407 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.124486 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:31.124767 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.125512 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.126814 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.127613 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.127691 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.127750 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.128086 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.128147 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260808 07:11:31.128422 27636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.123724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:42715 (local address 127.0.0.1:50470)
0808 07:11:31.124154 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.124197 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.124275 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.124596 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.124599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.124602 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:31.124676 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.124678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.125692 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.125693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.126708 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.126711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.127788 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.127790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.127882 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.127895 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.127964 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.128017 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":42,"thread_start_us":126,"threads_started":1}
I20260808 07:11:31.129299 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.123722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:43429 (local address 127.0.0.1:47404)
0808 07:11:31.124155 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.124200 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.124277 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.124614 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.124615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.124617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:31.124674 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.124676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.126951 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.126952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.128019 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.128021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.129034 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.129036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.129117 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.129121 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.129182 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.129225 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260808 07:11:31.129341 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.123919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.22.2.131:43429)
0808 07:11:31.124212 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.124218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.124268 (+ 50us) server_negotiation.cc:407] Connection header received
0808 07:11:31.124442 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.124444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.124461 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.124527 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:31.125616 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.126850 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.128117 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.128861 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.128940 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.128993 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.129212 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.129256 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":125,"threads_started":1}
I20260808 07:11:31.129220 27252 tablet_service.cc:1467] Processing CreateTablet for tablet e3543d57f3814f2e878f4b948c9cd171 (DEFAULT_TABLE table=test-table [id=5163889c57534ac1a6f8c53769a10505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:31.129446 27397 tablet_service.cc:1467] Processing CreateTablet for tablet e3543d57f3814f2e878f4b948c9cd171 (DEFAULT_TABLE table=test-table [id=5163889c57534ac1a6f8c53769a10505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:31.129622 27252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3543d57f3814f2e878f4b948c9cd171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:31.129719 27397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3543d57f3814f2e878f4b948c9cd171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:31.132328 27546 tablet_service.cc:1467] Processing CreateTablet for tablet e3543d57f3814f2e878f4b948c9cd171 (DEFAULT_TABLE table=test-table [id=5163889c57534ac1a6f8c53769a10505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:31.132457 27640 tablet_bootstrap.cc:492] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Bootstrap starting.
I20260808 07:11:31.132709 27546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3543d57f3814f2e878f4b948c9cd171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:31.132745 27641 tablet_bootstrap.cc:492] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568: Bootstrap starting.
I20260808 07:11:31.133041 27640 tablet_bootstrap.cc:654] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:31.133406 27641 tablet_bootstrap.cc:654] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:31.133490 27640 log.cc:824] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:31.134040 27641 log.cc:824] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:31.134315 27640 tablet_bootstrap.cc:492] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: No bootstrap required, opened a new log
I20260808 07:11:31.134382 27640 ts_tablet_manager.cc:1397] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:31.134905 27641 tablet_bootstrap.cc:492] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568: No bootstrap required, opened a new log
I20260808 07:11:31.134971 27641 ts_tablet_manager.cc:1397] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:31.136433 27645 tablet_bootstrap.cc:492] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Bootstrap starting.
I20260808 07:11:31.136479 27640 raft_consensus.cc:348] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.136646 27640 raft_consensus.cc:374] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:31.136687 27640 raft_consensus.cc:729] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47275513ef5d4119aec689383bc96b90, State: Initialized, Role: FOLLOWER
I20260808 07:11:31.136803 27640 consensus_queue.cc:260] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.137102 27640 ts_tablet_manager.cc:1428] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:31.137126 27472 heartbeater.cc:503] Master 127.22.2.190:35429 was elected leader, sending a full tablet report...
I20260808 07:11:31.137202 27641 raft_consensus.cc:348] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.137367 27641 raft_consensus.cc:374] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:31.137420 27641 raft_consensus.cc:729] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0f28a5b4d04b2ea58c083929894568, State: Initialized, Role: FOLLOWER
I20260808 07:11:31.137486 27645 tablet_bootstrap.cc:654] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:31.137549 27641 consensus_queue.cc:260] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.137858 27641 ts_tablet_manager.cc:1428] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:31.137887 27322 heartbeater.cc:503] Master 127.22.2.190:35429 was elected leader, sending a full tablet report...
I20260808 07:11:31.138087 27645 log.cc:824] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:31.138881 27645 tablet_bootstrap.cc:492] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: No bootstrap required, opened a new log
I20260808 07:11:31.138971 27645 ts_tablet_manager.cc:1397] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:11:31.140491 27645 raft_consensus.cc:348] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.140609 27645 raft_consensus.cc:374] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:31.140628 27645 raft_consensus.cc:729] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4746246f79f47f4835eb4ad109b9a6b, State: Initialized, Role: FOLLOWER
I20260808 07:11:31.140743 27645 consensus_queue.cc:260] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.141028 27645 ts_tablet_manager.cc:1428] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:31.141063 27620 heartbeater.cc:503] Master 127.22.2.190:35429 was elected leader, sending a full tablet report...
W20260808 07:11:31.168462 27473 tablet.cc:2369] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:31.265328 27323 tablet.cc:2369] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:31.334969 27649 raft_consensus.cc:482] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:31.335181 27649 raft_consensus.cc:504] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.335705 27649 leader_election.cc:288] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47275513ef5d4119aec689383bc96b90 (127.22.2.130:36665), ed0f28a5b4d04b2ea58c083929894568 (127.22.2.129:42715)
I20260808 07:11:31.338872 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.336109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:53199 (local address 127.22.2.129:42715)
0808 07:11:31.336162 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.336164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.336176 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:11:31.336214 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.336214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.336229 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.336307 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:31.336589 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.337077 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.338082 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.338498 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.338594 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.338603 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.338750 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.338792 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:11:31.338894 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.336071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:53569 (local address 127.22.2.130:36665)
0808 07:11:31.336129 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.336131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.336302 (+ 171us) server_negotiation.cc:407] Connection header received
0808 07:11:31.336344 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.336345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.336356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.336396 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:31.336598 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.337078 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.338127 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.338501 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.338591 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.338600 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.338786 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.338836 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:11:31.338886 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.335937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:42715 (local address 127.22.2.131:53199)
0808 07:11:31.336039 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.336052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.336109 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.336454 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.336455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.336456 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:31.336511 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.336512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.337191 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.337194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.337983 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.337985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.338592 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.338593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.338675 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.338678 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.338721 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.338782 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":45}
I20260808 07:11:31.338907 27650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.336002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:36665 (local address 127.22.2.131:53569)
0808 07:11:31.336221 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.336233 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.336295 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.336419 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.336419 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.336421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:31.336512 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.336513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.337165 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.337166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.337994 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.337996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.338600 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.338601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.338681 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.338684 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.338765 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.338804 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":139,"threads_started":1}
W20260808 07:11:31.339214 27621 tablet.cc:2369] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:31.339370 27274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3543d57f3814f2e878f4b948c9cd171" candidate_uuid: "a4746246f79f47f4835eb4ad109b9a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0f28a5b4d04b2ea58c083929894568" is_pre_election: true
I20260808 07:11:31.339565 27274 raft_consensus.cc:2454] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4746246f79f47f4835eb4ad109b9a6b in term 0.
I20260808 07:11:31.339596 27421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3543d57f3814f2e878f4b948c9cd171" candidate_uuid: "a4746246f79f47f4835eb4ad109b9a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47275513ef5d4119aec689383bc96b90" is_pre_election: true
I20260808 07:11:31.339748 27421 raft_consensus.cc:2454] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4746246f79f47f4835eb4ad109b9a6b in term 0.
I20260808 07:11:31.339861 27493 leader_election.cc:302] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4746246f79f47f4835eb4ad109b9a6b, ed0f28a5b4d04b2ea58c083929894568; no voters:
I20260808 07:11:31.340009 27649 raft_consensus.cc:2781] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:31.340073 27649 raft_consensus.cc:482] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:31.340130 27649 raft_consensus.cc:3037] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:31.340924 27649 raft_consensus.cc:504] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.341202 27649 leader_election.cc:288] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [CANDIDATE]: Term 1 election: Requested vote from peers 47275513ef5d4119aec689383bc96b90 (127.22.2.130:36665), ed0f28a5b4d04b2ea58c083929894568 (127.22.2.129:42715)
I20260808 07:11:31.341264 27421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3543d57f3814f2e878f4b948c9cd171" candidate_uuid: "a4746246f79f47f4835eb4ad109b9a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47275513ef5d4119aec689383bc96b90"
I20260808 07:11:31.341266 27274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3543d57f3814f2e878f4b948c9cd171" candidate_uuid: "a4746246f79f47f4835eb4ad109b9a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0f28a5b4d04b2ea58c083929894568"
I20260808 07:11:31.341354 27421 raft_consensus.cc:3037] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:31.341359 27274 raft_consensus.cc:3037] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:31.342080 27274 raft_consensus.cc:2454] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4746246f79f47f4835eb4ad109b9a6b in term 1.
I20260808 07:11:31.342276 27421 raft_consensus.cc:2454] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4746246f79f47f4835eb4ad109b9a6b in term 1.
I20260808 07:11:31.342291 27493 leader_election.cc:302] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4746246f79f47f4835eb4ad109b9a6b, ed0f28a5b4d04b2ea58c083929894568; no voters:
I20260808 07:11:31.342484 27649 raft_consensus.cc:2781] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:31.342694 27649 raft_consensus.cc:686] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 1 LEADER]: Becoming Leader. State: Replica: a4746246f79f47f4835eb4ad109b9a6b, State: Running, Role: LEADER
I20260808 07:11:31.342804 27649 consensus_queue.cc:237] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } }
I20260808 07:11:31.343705 27111 catalog_manager.cc:5495] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to a4746246f79f47f4835eb4ad109b9a6b (127.22.2.131). New cstate: current_term: 1 leader_uuid: "a4746246f79f47f4835eb4ad109b9a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4746246f79f47f4835eb4ad109b9a6b" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:31.358608 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.352577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.22.2.131:43429)
0808 07:11:31.352630 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.352633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.352643 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:31.355369 (+ 2726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.355370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.355415 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.355493 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:31.355709 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.356562 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.357748 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.357975 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.358052 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.358228 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:31.358356 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:31.358456 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.358475 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.358497 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:11:31.361735 27421 raft_consensus.cc:1261] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 1 FOLLOWER]: Refusing update from remote peer a4746246f79f47f4835eb4ad109b9a6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:31.361765 27274 raft_consensus.cc:1261] T e3543d57f3814f2e878f4b948c9cd171 P ed0f28a5b4d04b2ea58c083929894568 [term 1 FOLLOWER]: Refusing update from remote peer a4746246f79f47f4835eb4ad109b9a6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:31.362114 27649 consensus_queue.cc:1035] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "47275513ef5d4119aec689383bc96b90" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:31.362291 27651 consensus_queue.cc:1035] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0f28a5b4d04b2ea58c083929894568" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 42715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:31.367589 27111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:31.367722 27111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:31.368533 27111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c204cb5513744489d6204b8c6216fd1: Sending DeleteTablet for 3 replicas of tablet e3543d57f3814f2e878f4b948c9cd171
I20260808 07:11:31.368888 27546 tablet_service.cc:1514] Processing DeleteTablet for tablet e3543d57f3814f2e878f4b948c9cd171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:31 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:47404
I20260808 07:11:31.369052 27397 tablet_service.cc:1514] Processing DeleteTablet for tablet e3543d57f3814f2e878f4b948c9cd171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:31 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39276
I20260808 07:11:31.369120 27666 tablet_replica.cc:330] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: stopping tablet replica
I20260808 07:11:31.369216 27666 raft_consensus.cc:2229] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:31.369197 27252 tablet_service.cc:1514] Processing DeleteTablet for tablet e3543d57f3814f2e878f4b948c9cd171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:31 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:50470
I20260808 07:11:31.369336 27666 raft_consensus.cc:2258] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:31.369423 27667 tablet_replica.cc:330] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: stopping tablet replica
I20260808 07:11:31.369514 27667 raft_consensus.cc:2229] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:31.369613 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 27181
I20260808 07:11:31.369786 27667 raft_consensus.cc:2258] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:31.369860 27666 ts_tablet_manager.cc:1905] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:31.371824 27667 ts_tablet_manager.cc:1905] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:31.371902 27666 ts_tablet_manager.cc:1918] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:31.371968 27666 log.cc:1196] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/e3543d57f3814f2e878f4b948c9cd171
I20260808 07:11:31.372236 27666 ts_tablet_manager.cc:1939] T e3543d57f3814f2e878f4b948c9cd171 P a4746246f79f47f4835eb4ad109b9a6b: Deleting consensus metadata
I20260808 07:11:31.373314 27667 ts_tablet_manager.cc:1918] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:31.373375 27667 log.cc:1196] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/e3543d57f3814f2e878f4b948c9cd171
I20260808 07:11:31.373570 27667 ts_tablet_manager.cc:1939] T e3543d57f3814f2e878f4b948c9cd171 P 47275513ef5d4119aec689383bc96b90: Deleting consensus metadata
I20260808 07:11:31.374444 27097 catalog_manager.cc:4841] TS 47275513ef5d4119aec689383bc96b90 (127.22.2.130:36665): tablet e3543d57f3814f2e878f4b948c9cd171 (table test-table [id=5163889c57534ac1a6f8c53769a10505]) successfully deleted
I20260808 07:11:31.374513 27096 catalog_manager.cc:4841] TS a4746246f79f47f4835eb4ad109b9a6b (127.22.2.131:43429): tablet e3543d57f3814f2e878f4b948c9cd171 (table test-table [id=5163889c57534ac1a6f8c53769a10505]) successfully deleted
W20260808 07:11:31.380018 27095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:42715 (error 108)
I20260808 07:11:31.380340 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 27327
I20260808 07:11:31.380731 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.380543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:42715 (local address 127.0.0.1:50476)
0808 07:11:31.380592 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.380623 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:11:31.380877 27095 catalog_manager.cc:4584] TS ed0f28a5b4d04b2ea58c083929894568 (127.22.2.129:42715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3543d57f3814f2e878f4b948c9cd171: Network error: Client connection negotiation failed: client connection to 127.22.2.129:42715: connect: Connection refused (error 111)
I20260808 07:11:31.387249 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 27477
I20260808 07:11:31.394411 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 27078
2026-08-08T07:11:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2178 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T07:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:31Z Disabled control of system clock
I20260808 07:11:31.426498 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45173
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:45173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:11:31.508865 27675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:31.509073 27675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:31.509093 27675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:31.510161 27675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:31.510195 27675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:31.510207 27675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:31.510221 27675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:31.510232 27675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:31.511523 27675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:45173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:31.511742 27675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:31.511987 27675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:31.516839 27683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:31.516808 27681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:31.516795 27680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:31.516868 27675 server_base.cc:1034] running on GCE node
I20260808 07:11:31.517263 27675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:31.517560 27675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:31.518743 27675 hybrid_clock.cc:648] HybridClock initialized: now 1786173091518728 us; error 28 us; skew 500 ppm
I20260808 07:11:31.520012 27675 webserver.cc:460] Webserver started at http://127.22.2.190:45581/ using document root <none> and password file <none>
I20260808 07:11:31.520269 27675 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:31.520344 27675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:31.520500 27675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:31.521603 27675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "8e30c86c49d64acb8eca4430b7f95815"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "2673c289e2977bf72bd0785651b6c56b"
server_key_iv: "09695f02f14c4891ec2fac4abc777f79"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.521960 27675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "8e30c86c49d64acb8eca4430b7f95815"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "2673c289e2977bf72bd0785651b6c56b"
server_key_iv: "09695f02f14c4891ec2fac4abc777f79"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.523249 27675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:31.524035 27689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.524211 27675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.524325 27675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "8e30c86c49d64acb8eca4430b7f95815"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "2673c289e2977bf72bd0785651b6c56b"
server_key_iv: "09695f02f14c4891ec2fac4abc777f79"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.524401 27675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:31.527721 27675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:31.528002 27675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:31.528149 27675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:31.532109 27675 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:45173
I20260808 07:11:31.532140 27753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:45173 every 8 connection(s)
I20260808 07:11:31.532473 27675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:31.533196 27754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:31.536057 27754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815: Bootstrap starting.
I20260808 07:11:31.536636 27754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:31.536948 27754 log.cc:824] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:31.537631 27754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815: No bootstrap required, opened a new log
I20260808 07:11:31.539225 27754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e30c86c49d64acb8eca4430b7f95815" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45173 } }
I20260808 07:11:31.539363 27754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:31.539412 27754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e30c86c49d64acb8eca4430b7f95815, State: Initialized, Role: FOLLOWER
I20260808 07:11:31.539594 27754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e30c86c49d64acb8eca4430b7f95815" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45173 } }
I20260808 07:11:31.539688 27754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:31.539733 27754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:31.539788 27754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:31.540323 27754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e30c86c49d64acb8eca4430b7f95815" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45173 } }
I20260808 07:11:31.540442 27754 leader_election.cc:302] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e30c86c49d64acb8eca4430b7f95815; no voters:
I20260808 07:11:31.540629 27754 leader_election.cc:288] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:31.540655 27757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:31.540840 27757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [term 1 LEADER]: Becoming Leader. State: Replica: 8e30c86c49d64acb8eca4430b7f95815, State: Running, Role: LEADER
I20260808 07:11:31.540915 27754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:31.540928 27757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e30c86c49d64acb8eca4430b7f95815" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45173 } }
I20260808 07:11:31.541234 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 27675
I20260808 07:11:31.541359 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:31.541335 27760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e30c86c49d64acb8eca4430b7f95815. Latest consensus state: current_term: 1 leader_uuid: "8e30c86c49d64acb8eca4430b7f95815" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e30c86c49d64acb8eca4430b7f95815" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45173 } } }
I20260808 07:11:31.541406 27760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:31.541581 22538 external_mini_cluster.cc:1428] Setting key 0c59e8a3c8bd51dd01fa527c7b9cef41
I20260808 07:11:31.541922 27758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e30c86c49d64acb8eca4430b7f95815" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e30c86c49d64acb8eca4430b7f95815" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45173 } } }
I20260808 07:11:31.541999 27758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:31.542069 27768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:31.542865 27768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:31.544636 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.542310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.22.2.190:45173)
0808 07:11:31.542512 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.542522 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.542623 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:11:31.542783 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.542790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.542834 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.543062 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.544250 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.544253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.544253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:31.544257 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.544294 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:31.544305 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.544366 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.544460 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.544504 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260808 07:11:31.547968 27768 catalog_manager.cc:1347] Generated new cluster ID: ef97010d8fdd4da2af02a62d6c8b67f4
I20260808 07:11:31.548012 27768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:31.577044 27768 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:31.577252 27768 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:31.580731 27768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815: Generated new TSK 0
I20260808 07:11:31.580937 27768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:11:31.589088 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45173
--builtin_ntp_servers=127.22.2.148:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:11:31.689307 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:31.689560 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:31.689599 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:31.690795 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:31.690862 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:31.690907 27781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:31.692329 27781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:31.692617 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:31.692934 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:31.698208 27786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:31.698208 27789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:31.698228 27787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:31.698362 27781 server_base.cc:1034] running on GCE node
I20260808 07:11:31.698724 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:31.698987 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:31.700138 27781 hybrid_clock.cc:648] HybridClock initialized: now 1786173091700110 us; error 38 us; skew 500 ppm
I20260808 07:11:31.701386 27781 webserver.cc:460] Webserver started at http://127.22.2.129:44391/ using document root <none> and password file <none>
I20260808 07:11:31.701637 27781 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:31.701726 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:31.701890 27781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:31.702976 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "7aa3c3971d744ec2932427111e2417a1"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "2f927e2f60429925896963ea35793997"
server_key_iv: "b66e40e71638393c2159d9e3e1dacebc"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.703359 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "7aa3c3971d744ec2932427111e2417a1"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "2f927e2f60429925896963ea35793997"
server_key_iv: "b66e40e71638393c2159d9e3e1dacebc"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.704753 27781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:31.705526 27797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.705699 27781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:31.705801 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "7aa3c3971d744ec2932427111e2417a1"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "2f927e2f60429925896963ea35793997"
server_key_iv: "b66e40e71638393c2159d9e3e1dacebc"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.705885 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:31.709400 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:31.709681 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:31.709823 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:31.710043 27781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:31.710352 27781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:31.710405 27781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.710445 27781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:31.710471 27781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.716575 27781 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:41995
I20260808 07:11:31.716620 27931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:41995 every 8 connection(s)
I20260808 07:11:31.716944 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:31.719872 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.717670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:53791 (local address 127.22.2.190:45173)
0808 07:11:31.717718 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.717725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.717968 (+ 243us) server_negotiation.cc:407] Connection header received
0808 07:11:31.718443 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.718444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.718450 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.718515 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.719665 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.719667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.719667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:31.719671 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.719686 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:31.719691 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.719744 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.719804 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.719819 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:31.720063 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.717643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45173 (local address 127.22.2.129:53791)
0808 07:11:31.717837 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.717878 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.718356 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.718565 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.718568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.718576 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.719473 (+ 897us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:31.719473 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.719501 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:31.719529 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:31.719745 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:31.719748 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:31.719751 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.719806 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.719855 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260808 07:11:31.720562 27932 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45173
I20260808 07:11:31.720646 27932 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:31.720813 27932 heartbeater.cc:511] Master 127.22.2.190:45173 requested a full tablet report, sending...
I20260808 07:11:31.721390 27706 ts_manager.cc:194] Registered new tserver with Master: 7aa3c3971d744ec2932427111e2417a1 (127.22.2.129:41995)
I20260808 07:11:31.725265 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 27781
I20260808 07:11:31.725363 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:31.725507 22538 external_mini_cluster.cc:1428] Setting key 05b854054a68b30fa34349c01f5313bd
I20260808 07:11:31.726560 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45173
--builtin_ntp_servers=127.22.2.148:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:11:31.808575 27936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:31.808800 27936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:31.808823 27936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:31.810040 27936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:31.810079 27936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:31.810125 27936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:31.811447 27936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:31.811718 27936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:31.812012 27936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:31.817047 27936 server_base.cc:1034] running on GCE node
W20260808 07:11:31.816989 27945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:31.817008 27941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:31.816987 27942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:31.817507 27936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:31.817791 27936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:31.818964 27936 hybrid_clock.cc:648] HybridClock initialized: now 1786173091818958 us; error 35 us; skew 500 ppm
I20260808 07:11:31.820299 27936 webserver.cc:460] Webserver started at http://127.22.2.130:36499/ using document root <none> and password file <none>
I20260808 07:11:31.820536 27936 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:31.820605 27936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:31.820801 27936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:31.821882 27936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "e40359694ad7bb80f59a4bbc9e2985d6"
server_key_iv: "a6bae938bd55b47afae908b8fba1f55f"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.822259 27936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "e40359694ad7bb80f59a4bbc9e2985d6"
server_key_iv: "a6bae938bd55b47afae908b8fba1f55f"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.823637 27936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:31.824412 27953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.824585 27936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:31.824720 27936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "e40359694ad7bb80f59a4bbc9e2985d6"
server_key_iv: "a6bae938bd55b47afae908b8fba1f55f"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.824798 27936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:31.828264 27936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:31.828521 27936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:31.828660 27936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:31.828873 27936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:31.829172 27936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:31.829226 27936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.829265 27936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:31.829293 27936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.835197 27936 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:38631
I20260808 07:11:31.835260 28082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:38631 every 8 connection(s)
I20260808 07:11:31.835597 27936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:31.838852 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.836376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:37543 (local address 127.22.2.190:45173)
0808 07:11:31.836424 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.836426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.836742 (+ 316us) server_negotiation.cc:407] Connection header received
0808 07:11:31.837276 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.837278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.837285 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.837329 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.838602 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.838604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.838605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:31.838610 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.838627 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:31.838631 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.838687 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.838756 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.838771 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:31.839047 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.836340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45173 (local address 127.22.2.130:37543)
0808 07:11:31.836601 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.836647 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.837154 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.837416 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.837420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.837432 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.838443 (+ 1011us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:31.838443 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.838473 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:31.838502 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:31.838683 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:31.838688 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:31.838691 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.838758 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.838822 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
I20260808 07:11:31.839524 28083 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45173
I20260808 07:11:31.839676 28083 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:31.839838 28083 heartbeater.cc:511] Master 127.22.2.190:45173 requested a full tablet report, sending...
I20260808 07:11:31.840205 27706 ts_manager.cc:194] Registered new tserver with Master: 37d1f3c2ca2d4b3b9838d597785a7c7a (127.22.2.130:38631)
I20260808 07:11:31.841873 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 27936
I20260808 07:11:31.842005 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:31.842192 22538 external_mini_cluster.cc:1428] Setting key ce29734360fd91aadfb06196b403affc
I20260808 07:11:31.843103 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45173
--builtin_ntp_servers=127.22.2.148:34975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:11:31.931339 28087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:31.931598 28087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:31.931679 28087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:31.932955 28087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:31.932993 28087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:31.933023 28087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:31.934672 28087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:45173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:31.934902 28087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:31.935181 28087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:31.940683 28095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:31.940697 28092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:31.940910 28087 server_base.cc:1034] running on GCE node
W20260808 07:11:31.940704 28093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:31.941293 28087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:31.941579 28087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:31.942811 28087 hybrid_clock.cc:648] HybridClock initialized: now 1786173091942796 us; error 30 us; skew 500 ppm
I20260808 07:11:31.944108 28087 webserver.cc:460] Webserver started at http://127.22.2.131:39713/ using document root <none> and password file <none>
I20260808 07:11:31.944387 28087 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:31.944460 28087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:31.944620 28087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:31.945766 28087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "098e7cc28704440f8e4da1fa883b9afd"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "d700817f64eba0dd0e13e41046ed515f"
server_key_iv: "2bd3fc8826d75f4b26953c446c1d3ddd"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.946144 28087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "098e7cc28704440f8e4da1fa883b9afd"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "d700817f64eba0dd0e13e41046ed515f"
server_key_iv: "2bd3fc8826d75f4b26953c446c1d3ddd"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.947535 28087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:11:31.948441 28101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.948644 28087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:31.948758 28087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "098e7cc28704440f8e4da1fa883b9afd"
format_stamp: "Formatted at 2026-08-08 07:11:31 on dist-test-slave-6k22"
server_key: "d700817f64eba0dd0e13e41046ed515f"
server_key_iv: "2bd3fc8826d75f4b26953c446c1d3ddd"
server_key_version: "encryptionkey@0"
I20260808 07:11:31.948854 28087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:31.952669 28087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:31.952972 28087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:31.953133 28087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:31.953374 28087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:31.953724 28087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:31.953785 28087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.953830 28087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:31.953859 28087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:31.960661 28087 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:37911
I20260808 07:11:31.960707 28224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:37911 every 8 connection(s)
I20260808 07:11:31.961064 28087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:31.964470 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.961848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:54889 (local address 127.22.2.190:45173)
0808 07:11:31.961937 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.961940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.962131 (+ 191us) server_negotiation.cc:407] Connection header received
0808 07:11:31.962710 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.962712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.962720 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.962803 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.964217 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.964218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.964219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:31.964224 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.964242 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:31.964247 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.964312 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.964384 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.964399 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:11:31.964660 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.961759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:45173 (local address 127.22.2.131:54889)
0808 07:11:31.961961 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.962021 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.962525 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.962860 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.962864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.962872 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.964040 (+ 1168us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:31.964040 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.964072 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:31.964106 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:31.964311 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:31.964314 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:31.964316 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.964376 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.964428 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260808 07:11:31.965201 28225 heartbeater.cc:347] Connected to a master server at 127.22.2.190:45173
I20260808 07:11:31.965296 28225 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:31.965422 28225 heartbeater.cc:511] Master 127.22.2.190:45173 requested a full tablet report, sending...
I20260808 07:11:31.965871 27706 ts_manager.cc:194] Registered new tserver with Master: 098e7cc28704440f8e4da1fa883b9afd (127.22.2.131:37911)
I20260808 07:11:31.968212 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 28087
I20260808 07:11:31.968320 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:31.968467 22538 external_mini_cluster.cc:1428] Setting key fd2aab554ec18af72439ce3a6cc77b75
I20260808 07:11:31.969372 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:11:31.972080 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.970804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.22.2.190:45173)
0808 07:11:31.970876 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.970879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.970911 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:11:31.970994 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.970995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.971000 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.971051 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.971821 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.971823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.971823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:31.971828 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.971846 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:31.971851 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.971928 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.971986 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.972004 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:11:31.973839 27706 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:31.975631 27706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:31.980351 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.978436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:38631 (local address 127.0.0.1:53842)
0808 07:11:31.978934 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.978943 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.978981 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.979174 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.979181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.979189 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.979746 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:31.979746 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.979761 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:31.979781 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:31.980203 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:31.980204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:31.980205 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.980262 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.980277 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":75,"threads_started":1}
I20260808 07:11:31.980397 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.978559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53842 (local address 127.22.2.130:38631)
0808 07:11:31.978762 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.978768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.979035 (+ 267us) server_negotiation.cc:407] Connection header received
0808 07:11:31.979065 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.979067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.979076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.979150 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.979873 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.979883 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.979883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:31.979888 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.979925 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:31.979957 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.980020 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.980293 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.980310 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260808 07:11:31.980450 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.978299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50210 (local address 127.22.2.129:41995)
0808 07:11:31.978535 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.978540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.978816 (+ 276us) server_negotiation.cc:407] Connection header received
0808 07:11:31.978890 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.978894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.978906 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.978987 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.979991 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.980002 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.980002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:31.980007 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.980041 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:31.980083 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.980177 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.980315 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.980335 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":120,"threads_started":1}
I20260808 07:11:31.980682 28234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.978172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:41995 (local address 127.0.0.1:50210)
0808 07:11:31.978602 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.978739 (+ 137us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.978821 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.979376 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.979377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.979380 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.979898 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:31.979899 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.979907 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:31.979921 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:31.980180 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:31.980188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:31.980191 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.980610 (+ 419us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.980624 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":107,"threads_started":1}
I20260808 07:11:31.981086 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.978184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:37911 (local address 127.0.0.1:56120)
0808 07:11:31.978762 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.978773 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.978865 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.979402 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.979403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.979406 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.980079 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:31.980079 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.980086 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:31.980098 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:31.980359 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:31.980360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:31.980360 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 07:11:31.981016 (+ 656us) client_negotiation.cc:240] Negotiation successful
0808 07:11:31.981029 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":66,"threads_started":1}
I20260808 07:11:31.981151 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.978300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.22.2.131:37911)
0808 07:11:31.978535 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.978540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.978837 (+ 297us) server_negotiation.cc:407] Connection header received
0808 07:11:31.978948 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.978952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.978964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.979018 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.980188 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:31.980198 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:31.980198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:31.980203 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:31.980241 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:31.980271 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:31.980332 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:31.981039 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 07:11:31.981058 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":120,"threads_started":1}
I20260808 07:11:31.981215 28006 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf5720cb6743118ba04201465c2f7e (DEFAULT_TABLE table=test-table [id=b85c9c5dc3ef474e95beebf7fe9cdb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:31.981343 27850 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf5720cb6743118ba04201465c2f7e (DEFAULT_TABLE table=test-table [id=b85c9c5dc3ef474e95beebf7fe9cdb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:31.981640 27850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaf5720cb6743118ba04201465c2f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:31.981640 28006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaf5720cb6743118ba04201465c2f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:31.981969 28151 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf5720cb6743118ba04201465c2f7e (DEFAULT_TABLE table=test-table [id=b85c9c5dc3ef474e95beebf7fe9cdb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:31.982296 28151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaf5720cb6743118ba04201465c2f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:31.984953 28240 tablet_bootstrap.cc:492] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: Bootstrap starting.
I20260808 07:11:31.985275 28242 tablet_bootstrap.cc:492] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Bootstrap starting.
I20260808 07:11:31.985679 28240 tablet_bootstrap.cc:654] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:31.985915 28242 tablet_bootstrap.cc:654] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:31.986042 28240 log.cc:824] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:31.986261 28242 log.cc:824] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:31.986634 28241 tablet_bootstrap.cc:492] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1: Bootstrap starting.
I20260808 07:11:31.986804 28240 tablet_bootstrap.cc:492] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: No bootstrap required, opened a new log
I20260808 07:11:31.986871 28240 ts_tablet_manager.cc:1397] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:31.987077 28242 tablet_bootstrap.cc:492] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: No bootstrap required, opened a new log
I20260808 07:11:31.987157 28242 ts_tablet_manager.cc:1397] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:31.987320 28241 tablet_bootstrap.cc:654] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:31.987679 28241 log.cc:824] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:31.988480 28241 tablet_bootstrap.cc:492] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1: No bootstrap required, opened a new log
I20260808 07:11:31.988555 28241 ts_tablet_manager.cc:1397] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:11:31.989315 28240 raft_consensus.cc:348] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:31.989494 28240 raft_consensus.cc:374] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:31.989543 28240 raft_consensus.cc:729] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098e7cc28704440f8e4da1fa883b9afd, State: Initialized, Role: FOLLOWER
I20260808 07:11:31.989657 28240 consensus_queue.cc:260] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:31.989948 28240 ts_tablet_manager.cc:1428] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:31.989856 28242 raft_consensus.cc:348] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:31.990007 28242 raft_consensus.cc:374] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:31.990051 28242 raft_consensus.cc:729] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37d1f3c2ca2d4b3b9838d597785a7c7a, State: Initialized, Role: FOLLOWER
I20260808 07:11:31.990082 28225 heartbeater.cc:503] Master 127.22.2.190:45173 was elected leader, sending a full tablet report...
I20260808 07:11:31.990176 28242 consensus_queue.cc:260] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:31.990438 28242 ts_tablet_manager.cc:1428] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:11:31.990799 28083 heartbeater.cc:503] Master 127.22.2.190:45173 was elected leader, sending a full tablet report...
I20260808 07:11:31.991276 28241 raft_consensus.cc:348] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:31.991438 28241 raft_consensus.cc:374] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:31.991484 28241 raft_consensus.cc:729] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa3c3971d744ec2932427111e2417a1, State: Initialized, Role: FOLLOWER
I20260808 07:11:31.991622 28241 consensus_queue.cc:260] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:31.991919 28241 ts_tablet_manager.cc:1428] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:11:31.991961 27932 heartbeater.cc:503] Master 127.22.2.190:45173 was elected leader, sending a full tablet report...
I20260808 07:11:32.069298 28250 raft_consensus.cc:482] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:32.069490 28250 raft_consensus.cc:504] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:32.069886 28250 leader_election.cc:288] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098e7cc28704440f8e4da1fa883b9afd (127.22.2.131:37911), 37d1f3c2ca2d4b3b9838d597785a7c7a (127.22.2.130:38631)
I20260808 07:11:32.071698 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.070157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:37911 (local address 127.22.2.129:48157)
0808 07:11:32.070250 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.070265 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.070312 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.070534 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.070535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.070538 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:32.071270 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:32.071271 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:32.071274 (+ 3us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:32.071286 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:32.071521 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:32.071521 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:32.071522 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:11:32.071600 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:11:32.071617 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 07:11:32.071704 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.070237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48157 (local address 127.22.2.131:37911)
0808 07:11:32.070296 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.070298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.070324 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:11:32.070418 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.070420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.070429 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.070499 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.071408 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.071409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.071409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.071414 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.071433 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.071438 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.071509 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.071612 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.071628 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:11:32.071710 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.070157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:38631 (local address 127.22.2.129:44221)
0808 07:11:32.070429 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.070440 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.070557 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.070847 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.070848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.070851 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:32.071263 (+ 412us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:32.071263 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:32.071272 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:32.071286 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:32.071521 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:32.071521 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:32.071522 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:11:32.071620 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:11:32.071630 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":82,"thread_start_us":119,"threads_started":1}
I20260808 07:11:32.071772 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.070289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:44221 (local address 127.22.2.130:38631)
0808 07:11:32.070351 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.070353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.070536 (+ 183us) server_negotiation.cc:407] Connection header received
0808 07:11:32.070732 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.070733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.070740 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.070812 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.071407 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.071409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.071409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.071414 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.071432 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.071437 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.071509 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.071626 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.071639 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:11:32.072211 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf5720cb6743118ba04201465c2f7e" candidate_uuid: "7aa3c3971d744ec2932427111e2417a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" is_pre_election: true
I20260808 07:11:32.072399 28030 raft_consensus.cc:2454] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa3c3971d744ec2932427111e2417a1 in term 0.
I20260808 07:11:32.072355 28174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf5720cb6743118ba04201465c2f7e" candidate_uuid: "7aa3c3971d744ec2932427111e2417a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098e7cc28704440f8e4da1fa883b9afd" is_pre_election: true
I20260808 07:11:32.072526 28174 raft_consensus.cc:2454] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa3c3971d744ec2932427111e2417a1 in term 0.
I20260808 07:11:32.072659 27798 leader_election.cc:302] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37d1f3c2ca2d4b3b9838d597785a7c7a, 7aa3c3971d744ec2932427111e2417a1; no voters:
I20260808 07:11:32.072824 28250 raft_consensus.cc:2781] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:32.072892 28250 raft_consensus.cc:482] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:32.072918 28250 raft_consensus.cc:3037] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.073665 28250 raft_consensus.cc:504] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:32.073817 28250 leader_election.cc:288] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [CANDIDATE]: Term 1 election: Requested vote from peers 098e7cc28704440f8e4da1fa883b9afd (127.22.2.131:37911), 37d1f3c2ca2d4b3b9838d597785a7c7a (127.22.2.130:38631)
I20260808 07:11:32.073990 28174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf5720cb6743118ba04201465c2f7e" candidate_uuid: "7aa3c3971d744ec2932427111e2417a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098e7cc28704440f8e4da1fa883b9afd"
I20260808 07:11:32.074056 28174 raft_consensus.cc:3037] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.074749 28174 raft_consensus.cc:2454] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa3c3971d744ec2932427111e2417a1 in term 1.
I20260808 07:11:32.074874 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf5720cb6743118ba04201465c2f7e" candidate_uuid: "7aa3c3971d744ec2932427111e2417a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a"
I20260808 07:11:32.074959 27799 leader_election.cc:302] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098e7cc28704440f8e4da1fa883b9afd, 7aa3c3971d744ec2932427111e2417a1; no voters:
I20260808 07:11:32.074966 28030 raft_consensus.cc:3037] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.075032 28250 raft_consensus.cc:2781] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:32.075171 28250 raft_consensus.cc:686] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa3c3971d744ec2932427111e2417a1, State: Running, Role: LEADER
I20260808 07:11:32.075270 28250 consensus_queue.cc:237] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } }
I20260808 07:11:32.075944 28030 raft_consensus.cc:2454] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa3c3971d744ec2932427111e2417a1 in term 1.
I20260808 07:11:32.076193 27705 catalog_manager.cc:5495] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa3c3971d744ec2932427111e2417a1 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "7aa3c3971d744ec2932427111e2417a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa3c3971d744ec2932427111e2417a1" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 41995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 } health_report { overall_health: UNKNOWN } } }
W20260808 07:11:32.086313 28084 tablet.cc:2369] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:32.097277 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.096332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.22.2.129:41995)
0808 07:11:32.096399 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.096401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.096411 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:32.096458 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.096459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.096472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.096513 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.097083 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.097084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.097084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.097088 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.097103 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.097107 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.097153 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.097212 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.097224 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:11:32.099077 28030 raft_consensus.cc:1261] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 1 FOLLOWER]: Refusing update from remote peer 7aa3c3971d744ec2932427111e2417a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:32.099112 28174 raft_consensus.cc:1261] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 1 FOLLOWER]: Refusing update from remote peer 7aa3c3971d744ec2932427111e2417a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:32.099452 28252 consensus_queue.cc:1035] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37d1f3c2ca2d4b3b9838d597785a7c7a" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:32.099599 28250 consensus_queue.cc:1035] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098e7cc28704440f8e4da1fa883b9afd" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:32.102078 28255 mvcc.cc:205] Tried to move back new op lower bound from 7316164985235152896 to 7316164985140867072. Current Snapshot: MvccSnapshot[applied={T|T < 7316164985235152896}]
I20260808 07:11:32.102385 28257 mvcc.cc:205] Tried to move back new op lower bound from 7316164985235152896 to 7316164985140867072. Current Snapshot: MvccSnapshot[applied={T|T < 7316164985235152896}]
I20260808 07:11:32.103341 28256 mvcc.cc:205] Tried to move back new op lower bound from 7316164985235152896 to 7316164985140867072. Current Snapshot: MvccSnapshot[applied={T|T < 7316164985235152896}]
I20260808 07:11:32.104460 27706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:32.104576 27706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:32.105369 27706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e30c86c49d64acb8eca4430b7f95815: Sending DeleteTablet for 3 replicas of tablet fdaf5720cb6743118ba04201465c2f7e
I20260808 07:11:32.105793 28151 tablet_service.cc:1514] Processing DeleteTablet for tablet fdaf5720cb6743118ba04201465c2f7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:32 UTC) from {username='slave'} at 127.0.0.1:56120
I20260808 07:11:32.105861 28006 tablet_service.cc:1514] Processing DeleteTablet for tablet fdaf5720cb6743118ba04201465c2f7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:32 UTC) from {username='slave'} at 127.0.0.1:53842
I20260808 07:11:32.106092 28265 tablet_replica.cc:330] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: stopping tablet replica
I20260808 07:11:32.106133 27850 tablet_service.cc:1514] Processing DeleteTablet for tablet fdaf5720cb6743118ba04201465c2f7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:32 UTC) from {username='slave'} at 127.0.0.1:50210
I20260808 07:11:32.106208 28265 raft_consensus.cc:2229] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:32.106312 28265 raft_consensus.cc:2258] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:32.106438 28266 tablet_replica.cc:330] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1: stopping tablet replica
I20260808 07:11:32.106576 28266 raft_consensus.cc:2229] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:32.106629 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 27781
I20260808 07:11:32.106756 28266 raft_consensus.cc:2258] T fdaf5720cb6743118ba04201465c2f7e P 7aa3c3971d744ec2932427111e2417a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:32.109462 28264 tablet_replica.cc:330] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: stopping tablet replica
I20260808 07:11:32.109587 28264 raft_consensus.cc:2229] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:32.109700 28264 raft_consensus.cc:2258] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:32.110275 28264 ts_tablet_manager.cc:1905] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:32.111696 28265 ts_tablet_manager.cc:1905] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:32.112546 28264 ts_tablet_manager.cc:1918] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:32.112646 28264 log.cc:1196] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/fdaf5720cb6743118ba04201465c2f7e
I20260808 07:11:32.112951 28264 ts_tablet_manager.cc:1939] T fdaf5720cb6743118ba04201465c2f7e P 098e7cc28704440f8e4da1fa883b9afd: Deleting consensus metadata
I20260808 07:11:32.113232 28265 ts_tablet_manager.cc:1918] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:32.113325 28265 log.cc:1196] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/fdaf5720cb6743118ba04201465c2f7e
I20260808 07:11:32.113543 27691 catalog_manager.cc:4841] TS 098e7cc28704440f8e4da1fa883b9afd (127.22.2.131:37911): tablet fdaf5720cb6743118ba04201465c2f7e (table test-table [id=b85c9c5dc3ef474e95beebf7fe9cdb7f]) successfully deleted
I20260808 07:11:32.113628 28265 ts_tablet_manager.cc:1939] T fdaf5720cb6743118ba04201465c2f7e P 37d1f3c2ca2d4b3b9838d597785a7c7a: Deleting consensus metadata
I20260808 07:11:32.114022 27690 catalog_manager.cc:4841] TS 37d1f3c2ca2d4b3b9838d597785a7c7a (127.22.2.130:38631): tablet fdaf5720cb6743118ba04201465c2f7e (table test-table [id=b85c9c5dc3ef474e95beebf7fe9cdb7f]) successfully deleted
W20260808 07:11:32.115550 27690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:41995 (error 108)
I20260808 07:11:32.115849 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 27936
I20260808 07:11:32.116940 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.115986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:41995 (local address 127.0.0.1:50234)
0808 07:11:32.116852 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.116898 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:41995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":844}
W20260808 07:11:32.117936 27690 catalog_manager.cc:4584] TS 7aa3c3971d744ec2932427111e2417a1 (127.22.2.129:41995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fdaf5720cb6743118ba04201465c2f7e: Network error: Client connection negotiation failed: client connection to 127.22.2.129:41995: connect: Connection refused (error 111)
I20260808 07:11:32.121860 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 28087
I20260808 07:11:32.130707 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 27675
2026-08-08T07:11:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (736 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 07:11:32.154839 22538 mini_oidc.cc:152] Starting JWKS server
I20260808 07:11:32.156021 22538 webserver.cc:460] Webserver started at https://127.0.0.1:44751/ using document root <none> and password file <none>
I20260808 07:11:32.156106 22538 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:11:32.156265 22538 webserver.cc:460] Webserver started at http://127.0.0.1:45145/ using document root <none> and password file <none>
2026-08-08T07:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:32Z Disabled control of system clock
I20260808 07:11:32.164841 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40049
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:36247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40049
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44751/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/testchainca.cert with env {}
W20260808 07:11:32.249357 28274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.249593 28274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.249630 28274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.250787 28274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:32.250856 28274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.250878 28274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.250898 28274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:32.250928 28274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:11:32.250958 28274 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:11:32.250985 28274 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44751/jwks.json
I20260808 07:11:32.252727 28274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:36247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40049
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44751/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:32.253006 28274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.253233 28274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:32.258448 28274 server_base.cc:1034] running on GCE node
W20260808 07:11:32.258424 28279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.258427 28281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.258399 28283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.258991 28274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.259325 28274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.260587 28274 hybrid_clock.cc:648] HybridClock initialized: now 1786173092260551 us; error 62 us; skew 500 ppm
I20260808 07:11:32.262076 28274 webserver.cc:460] Webserver started at http://127.22.2.190:42889/ using document root <none> and password file <none>
I20260808 07:11:32.262372 28274 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.262451 28274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.262604 28274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.263738 28274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "3c39b4ec29e84d268ed6cd5b88747208"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "1f6c9d9723638704d366f0c4573c2b67"
server_key_iv: "f7b7c6910d2c4f8fb595a02ec173ea92"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.264093 28274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "3c39b4ec29e84d268ed6cd5b88747208"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "1f6c9d9723638704d366f0c4573c2b67"
server_key_iv: "f7b7c6910d2c4f8fb595a02ec173ea92"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.265398 28274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:11:32.266170 28290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.266333 28274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:32.266443 28274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "3c39b4ec29e84d268ed6cd5b88747208"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "1f6c9d9723638704d366f0c4573c2b67"
server_key_iv: "f7b7c6910d2c4f8fb595a02ec173ea92"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.266518 28274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.295161 28274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.295544 28274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.295756 28274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.299901 28274 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:40049
I20260808 07:11:32.299937 28348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:40049 every 8 connection(s)
I20260808 07:11:32.300310 28274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:32.301276 28349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:32.303645 28349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208: Bootstrap starting.
I20260808 07:11:32.304283 28349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:32.304591 28349 log.cc:824] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:32.305308 28349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208: No bootstrap required, opened a new log
I20260808 07:11:32.306911 28349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c39b4ec29e84d268ed6cd5b88747208" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40049 } }
I20260808 07:11:32.307049 28349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:32.307071 28349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c39b4ec29e84d268ed6cd5b88747208, State: Initialized, Role: FOLLOWER
I20260808 07:11:32.307232 28349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c39b4ec29e84d268ed6cd5b88747208" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40049 } }
I20260808 07:11:32.307298 28349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:32.307322 28349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:32.307392 28349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.307977 28349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c39b4ec29e84d268ed6cd5b88747208" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40049 } }
I20260808 07:11:32.308084 28349 leader_election.cc:302] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c39b4ec29e84d268ed6cd5b88747208; no voters:
I20260808 07:11:32.308274 28349 leader_election.cc:288] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:32.308332 28352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:32.308511 28352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [term 1 LEADER]: Becoming Leader. State: Replica: 3c39b4ec29e84d268ed6cd5b88747208, State: Running, Role: LEADER
I20260808 07:11:32.308552 28349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:32.308632 28352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c39b4ec29e84d268ed6cd5b88747208" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40049 } }
I20260808 07:11:32.309038 28354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c39b4ec29e84d268ed6cd5b88747208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c39b4ec29e84d268ed6cd5b88747208" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40049 } } }
I20260808 07:11:32.309139 28354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.309108 28355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c39b4ec29e84d268ed6cd5b88747208. Latest consensus state: current_term: 1 leader_uuid: "3c39b4ec29e84d268ed6cd5b88747208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c39b4ec29e84d268ed6cd5b88747208" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40049 } } }
I20260808 07:11:32.309345 28355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.309399 28362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:32.309886 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 28274
I20260808 07:11:32.309913 28362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:32.310007 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:32.310230 22538 external_mini_cluster.cc:1428] Setting key 3546b7bd0949ad2ef94cdaee7d16014d
I20260808 07:11:32.311791 28362 catalog_manager.cc:1347] Generated new cluster ID: fb01d17873db43f7a032daf3681c5b7e
I20260808 07:11:32.311862 28362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:32.316537 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.310754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.22.2.190:40049)
0808 07:11:32.311001 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.311009 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.311039 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:11:32.311163 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.311169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.311255 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.311419 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.311727 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.313103 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.314218 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.314902 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.315034 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.315767 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.315771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.315772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.315990 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.316033 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.316047 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.316133 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.316209 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.316292 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260808 07:11:32.317847 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:11:32.325410 28362 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:32.325930 28362 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:32.330163 28362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c39b4ec29e84d268ed6cd5b88747208: Generated new TSK 0
I20260808 07:11:32.330355 28362 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:11:32.354887 28383 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 07:11:32.355043 28383 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 07:11:32.355288 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.339122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.22.2.190:40049)
0808 07:11:32.339180 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.339182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.339301 (+ 119us) server_negotiation.cc:407] Connection header received
0808 07:11:32.339385 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.339386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.339399 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.339485 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:11:32.339634 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.340453 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.341441 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.341764 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.341838 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.342051 (+ 213us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:11:32.355028 (+ 12977us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:11:32.355205 (+ 177us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44900: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 07:11:32.355350 28372 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44900: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 07:11:32.355717 28381 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:32.339055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40049 (local address 127.0.0.1:44900)
0808 07:11:32.339231 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.339241 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.339296 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.339485 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.339486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.339487 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:11:32.339543 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.339545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.340730 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.340731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.341340 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.341343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.341866 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.341867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.341911 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.341930 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:11:32.355229 (+ 13299us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:11:32.355361 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.2.190:40049: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.2.190:40049: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44751/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 07:11:32.356201 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 28274
2026-08-08T07:11:32Z chronyd exiting
I20260808 07:11:32.396037 22538 test_util.cc:175] -----------------------------------------------
I20260808 07:11:32.396148 22538 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173068742558-22538-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (242 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 07:11:32.396917 22538 mini_oidc.cc:152] Starting JWKS server
I20260808 07:11:32.397634 22538 webserver.cc:460] Webserver started at https://127.0.0.1:44279/ using document root <none> and password file <none>
I20260808 07:11:32.397703 22538 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:11:32.397893 22538 webserver.cc:460] Webserver started at http://127.0.0.1:40111/ using document root <none> and password file <none>
2026-08-08T07:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:32Z Disabled control of system clock
I20260808 07:11:32.406462 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42173
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42173
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44279/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/testchainca.cert with env {}
W20260808 07:11:32.488317 28389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.488517 28389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.488540 28389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.489708 28389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:32.489754 28389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.489768 28389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.489780 28389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:32.489792 28389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:11:32.489804 28389 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:11:32.489858 28389 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44279/jwks.json
I20260808 07:11:32.491358 28389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44279/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:32.491636 28389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.491855 28389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:32.496803 28395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.496773 28398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.496790 28394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.497138 28389 server_base.cc:1034] running on GCE node
I20260808 07:11:32.497438 28389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.497743 28389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.498907 28389 hybrid_clock.cc:648] HybridClock initialized: now 1786173092498886 us; error 42 us; skew 500 ppm
I20260808 07:11:32.500267 28389 webserver.cc:460] Webserver started at http://127.22.2.190:37653/ using document root <none> and password file <none>
I20260808 07:11:32.500536 28389 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.500609 28389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.500756 28389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.501816 28389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "436ecba9009148febfddd7c9936fa27c"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "8129a0e597be478d7608e7d33ba1bf63"
server_key_iv: "b73801de8d32b6cf7d060b8eb681c1c9"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.502184 28389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "436ecba9009148febfddd7c9936fa27c"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "8129a0e597be478d7608e7d33ba1bf63"
server_key_iv: "b73801de8d32b6cf7d060b8eb681c1c9"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.503511 28389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:32.504405 28406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.504580 28389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:32.504680 28389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "436ecba9009148febfddd7c9936fa27c"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "8129a0e597be478d7608e7d33ba1bf63"
server_key_iv: "b73801de8d32b6cf7d060b8eb681c1c9"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.504768 28389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.513175 28389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.513501 28389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.513653 28389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.518102 28389 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42173
I20260808 07:11:32.518136 28467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42173 every 8 connection(s)
I20260808 07:11:32.518533 28389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:32.519241 28468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:32.520934 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 28389
I20260808 07:11:32.521034 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:32.521243 22538 external_mini_cluster.cc:1428] Setting key ab038acfbd946da75c22cdf9118b9549
I20260808 07:11:32.522661 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c: Bootstrap starting.
I20260808 07:11:32.523303 28468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:32.523669 28468 log.cc:824] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:32.524427 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c: No bootstrap required, opened a new log
I20260808 07:11:32.526767 28468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436ecba9009148febfddd7c9936fa27c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42173 } }
I20260808 07:11:32.526943 28468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:32.526974 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.521749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58318 (local address 127.22.2.190:42173)
0808 07:11:32.521949 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.521956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.521982 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:11:32.522396 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.522402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.522458 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.522573 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.522860 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.523926 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.524849 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.525461 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.525587 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.526250 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.526255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.526256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.526507 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.526548 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.526562 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.526633 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.526695 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.526775 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260808 07:11:32.526984 28468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 436ecba9009148febfddd7c9936fa27c, State: Initialized, Role: FOLLOWER
I20260808 07:11:32.527139 28468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436ecba9009148febfddd7c9936fa27c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42173 } }
I20260808 07:11:32.527225 28468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:32.527261 28468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:32.527309 28468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.528026 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:11:32.528021 28468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436ecba9009148febfddd7c9936fa27c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42173 } }
I20260808 07:11:32.528144 28468 leader_election.cc:302] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 436ecba9009148febfddd7c9936fa27c; no voters:
I20260808 07:11:32.528322 28468 leader_election.cc:288] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:32.528373 28473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:32.528502 28473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [term 1 LEADER]: Becoming Leader. State: Replica: 436ecba9009148febfddd7c9936fa27c, State: Running, Role: LEADER
I20260808 07:11:32.528601 28468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:32.528673 28473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436ecba9009148febfddd7c9936fa27c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42173 } }
I20260808 07:11:32.529040 28473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 436ecba9009148febfddd7c9936fa27c. Latest consensus state: current_term: 1 leader_uuid: "436ecba9009148febfddd7c9936fa27c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436ecba9009148febfddd7c9936fa27c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42173 } } }
I20260808 07:11:32.529138 28473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.529373 28484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:32.529924 28484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:32.530259 28474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "436ecba9009148febfddd7c9936fa27c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "436ecba9009148febfddd7c9936fa27c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42173 } } }
I20260808 07:11:32.530349 28474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.531774 28484 catalog_manager.cc:1347] Generated new cluster ID: af70b48859bd49ab849dbd0a3a4a3844
I20260808 07:11:32.531828 28484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:32.548437 28484 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:32.549168 28484 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:32.557006 28484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 436ecba9009148febfddd7c9936fa27c: Generated new TSK 0
I20260808 07:11:32.557219 28484 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:11:32.586706 28503 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 07:11:32.586812 28503 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 07:11:32.587117 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.564760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58322 (local address 127.22.2.190:42173)
0808 07:11:32.564808 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.564810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.564920 (+ 110us) server_negotiation.cc:407] Connection header received
0808 07:11:32.564979 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.564980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.564991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.565048 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:11:32.565183 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.565681 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.566837 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.567023 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.567073 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.567259 (+ 186us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:11:32.586879 (+ 19620us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:11:32.587048 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58322: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:11:32.587195 28471 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58322: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 07:11:32.587253 28502 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:32.564655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42173 (local address 127.0.0.1:58322)
0808 07:11:32.564836 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.564849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.564918 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.565058 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.565059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.565061 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:11:32.565120 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.565122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.565772 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.565773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.566747 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.566750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.567096 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.567097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.567159 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.567165 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:11:32.587025 (+ 19860us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:11:32.587149 (+ 124us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.2.190:42173: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
I20260808 07:11:32.587466 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 28389
2026-08-08T07:11:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (231 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 07:11:32.628767 22538 mini_oidc.cc:152] Starting JWKS server
I20260808 07:11:32.629485 22538 webserver.cc:460] Webserver started at https://127.0.0.1:46233/ using document root <none> and password file <none>
I20260808 07:11:32.629556 22538 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:11:32.629727 22538 webserver.cc:460] Webserver started at http://127.0.0.1:36831/ using document root <none> and password file <none>
2026-08-08T07:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:32Z Disabled control of system clock
I20260808 07:11:32.638316 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:43421
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:35955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:43421
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46233/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:11:32.720872 28511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.721084 28511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.721105 28511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.722330 28511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:32.722378 28511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.722391 28511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.722404 28511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:32.722448 28511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:11:32.722478 28511 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:11:32.722494 28511 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46233/jwks.json
I20260808 07:11:32.723913 28511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:43421
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:43421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46233/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:32.724184 28511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.724494 28511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:32.729580 28518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.729794 28511 server_base.cc:1034] running on GCE node
W20260808 07:11:32.729614 28522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.729621 28519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.730121 28511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.730432 28511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.731606 28511 hybrid_clock.cc:648] HybridClock initialized: now 1786173092731590 us; error 37 us; skew 500 ppm
I20260808 07:11:32.732913 28511 webserver.cc:460] Webserver started at http://127.22.2.190:42859/ using document root <none> and password file <none>
I20260808 07:11:32.733173 28511 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.733220 28511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.733361 28511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.734450 28511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "de6b32bff0174512bc4054c87a466b48"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "9fed0b5e73f74e89bf5711f81fec8304"
server_key_iv: "e8e26320ff3d24d58849396d1f87e01b"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.734822 28511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "de6b32bff0174512bc4054c87a466b48"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "9fed0b5e73f74e89bf5711f81fec8304"
server_key_iv: "e8e26320ff3d24d58849396d1f87e01b"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.736351 28511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:32.737105 28530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.737295 28511 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:32.737396 28511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "de6b32bff0174512bc4054c87a466b48"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "9fed0b5e73f74e89bf5711f81fec8304"
server_key_iv: "e8e26320ff3d24d58849396d1f87e01b"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.737453 28511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.749328 28511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.749596 28511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.749706 28511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.753796 28511 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:43421
I20260808 07:11:32.753829 28593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:43421 every 8 connection(s)
I20260808 07:11:32.754211 28511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:32.754909 28595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:32.757712 28595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48: Bootstrap starting.
I20260808 07:11:32.758285 28595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:32.758605 28595 log.cc:824] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:32.759337 28595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48: No bootstrap required, opened a new log
I20260808 07:11:32.760957 28595 raft_consensus.cc:348] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6b32bff0174512bc4054c87a466b48" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43421 } }
I20260808 07:11:32.761119 28595 raft_consensus.cc:374] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:32.761171 28595 raft_consensus.cc:729] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6b32bff0174512bc4054c87a466b48, State: Initialized, Role: FOLLOWER
I20260808 07:11:32.761354 28595 consensus_queue.cc:260] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6b32bff0174512bc4054c87a466b48" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43421 } }
I20260808 07:11:32.761454 28595 raft_consensus.cc:388] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:32.761499 28595 raft_consensus.cc:482] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:32.761549 28595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.762082 28595 raft_consensus.cc:504] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6b32bff0174512bc4054c87a466b48" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43421 } }
I20260808 07:11:32.762198 28595 leader_election.cc:302] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de6b32bff0174512bc4054c87a466b48; no voters:
I20260808 07:11:32.762398 28595 leader_election.cc:288] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:32.762465 28598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:32.762655 28595 sys_catalog.cc:564] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:32.762717 28598 raft_consensus.cc:686] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [term 1 LEADER]: Becoming Leader. State: Replica: de6b32bff0174512bc4054c87a466b48, State: Running, Role: LEADER
I20260808 07:11:32.762872 28598 consensus_queue.cc:237] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6b32bff0174512bc4054c87a466b48" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43421 } }
I20260808 07:11:32.763211 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 28511
I20260808 07:11:32.763336 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:32.763373 28601 sys_catalog.cc:455] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de6b32bff0174512bc4054c87a466b48. Latest consensus state: current_term: 1 leader_uuid: "de6b32bff0174512bc4054c87a466b48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6b32bff0174512bc4054c87a466b48" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43421 } } }
I20260808 07:11:32.763468 28601 sys_catalog.cc:458] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.763706 22538 external_mini_cluster.cc:1428] Setting key b5c7217459dd64a3957d3bd235c6a92e
I20260808 07:11:32.763908 28600 sys_catalog.cc:455] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de6b32bff0174512bc4054c87a466b48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6b32bff0174512bc4054c87a466b48" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 43421 } } }
I20260808 07:11:32.764001 28600 sys_catalog.cc:458] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.764302 28612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:32.764930 28612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:32.766752 28612 catalog_manager.cc:1347] Generated new cluster ID: e81126537af6471ba6f94a85b8066a30
I20260808 07:11:32.766820 28612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:32.769349 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.764353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.22.2.190:43421)
0808 07:11:32.764539 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.764548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.764685 (+ 137us) server_negotiation.cc:407] Connection header received
0808 07:11:32.764800 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.764812 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.764863 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.765009 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.765243 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.766411 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.767337 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.767854 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.768006 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.768683 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.768687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.768687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.768873 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.768905 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.768915 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.768990 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.769061 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.769139 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260808 07:11:32.770135 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:11:32.781951 28612 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:32.782605 28612 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:32.787142 28612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de6b32bff0174512bc4054c87a466b48: Generated new TSK 0
I20260808 07:11:32.787298 28612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:11:32.797220 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.793613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.22.2.190:43421)
0808 07:11:32.793658 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.793660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.793736 (+ 76us) server_negotiation.cc:407] Connection header received
0808 07:11:32.793827 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.793828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.793838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.793896 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:11:32.794048 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.794535 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.795777 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.795971 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.796025 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.796227 (+ 202us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:11:32.796986 (+ 759us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46233/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:11:32.797139 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37308: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46233/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 07:11:32.797297 28631 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:32.793514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:43421 (local address 127.0.0.1:37308)
0808 07:11:32.793677 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.793688 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.793757 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.793914 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.793915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.793917 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:11:32.793979 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.793982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.794637 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.794639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.795697 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.795701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.796050 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.796052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.796121 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.796128 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:11:32.797091 (+ 963us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46233/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:11:32.797184 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.2.190:43421: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46233/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
W20260808 07:11:32.797290 28621 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37308: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46233/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260808 07:11:32.797672 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 28511
E20260808 07:11:32.809098 28632 webserver.cc:533] Webserver: sslize failed:
E20260808 07:11:32.809178 28632 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T07:11:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (208 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T07:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:32Z Disabled control of system clock
I20260808 07:11:32.846649 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41845
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41845
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/testchainca.cert with env {}
W20260808 07:11:32.929278 28636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.929493 28636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.929517 28636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.930603 28636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:32.930636 28636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.930649 28636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.930661 28636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:32.930673 28636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:11:32.930685 28636 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:11:32.930748 28636 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 07:11:32.932320 28636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41845
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:32.932598 28636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.932832 28636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:32.938030 28645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.938043 28642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.938218 28643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.938328 28636 server_base.cc:1034] running on GCE node
I20260808 07:11:32.938596 28636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.938954 28636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.940120 28636 hybrid_clock.cc:648] HybridClock initialized: now 1786173092940101 us; error 53 us; skew 500 ppm
I20260808 07:11:32.941462 28636 webserver.cc:460] Webserver started at http://127.22.2.190:41861/ using document root <none> and password file <none>
I20260808 07:11:32.941730 28636 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.941805 28636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.941957 28636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.943037 28636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "be4c597573ad4bc496ed19f7e09caa0a"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "70613952cbb256a251881a30a947a7d1"
server_key_iv: "19825edcdca60ca26c7d638e86d28a07"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.943426 28636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "be4c597573ad4bc496ed19f7e09caa0a"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "70613952cbb256a251881a30a947a7d1"
server_key_iv: "19825edcdca60ca26c7d638e86d28a07"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.944921 28636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:32.945776 28653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.945969 28636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:32.946075 28636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "be4c597573ad4bc496ed19f7e09caa0a"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-6k22"
server_key: "70613952cbb256a251881a30a947a7d1"
server_key_iv: "19825edcdca60ca26c7d638e86d28a07"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.946175 28636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.968158 28636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.968519 28636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.968681 28636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.973434 28636 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41845
I20260808 07:11:32.973472 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41845 every 8 connection(s)
I20260808 07:11:32.973853 28636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:32.974933 28714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:32.977283 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a: Bootstrap starting.
I20260808 07:11:32.977866 28714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:32.978112 28714 log.cc:824] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:32.978822 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a: No bootstrap required, opened a new log
I20260808 07:11:32.980582 28714 raft_consensus.cc:348] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4c597573ad4bc496ed19f7e09caa0a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41845 } }
I20260808 07:11:32.980728 28714 raft_consensus.cc:374] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:32.980752 28714 raft_consensus.cc:729] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be4c597573ad4bc496ed19f7e09caa0a, State: Initialized, Role: FOLLOWER
I20260808 07:11:32.980903 28714 consensus_queue.cc:260] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4c597573ad4bc496ed19f7e09caa0a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41845 } }
I20260808 07:11:32.980990 28714 raft_consensus.cc:388] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:32.981014 28714 raft_consensus.cc:482] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:32.981045 28714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.981591 28714 raft_consensus.cc:504] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4c597573ad4bc496ed19f7e09caa0a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41845 } }
I20260808 07:11:32.981603 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 28636
I20260808 07:11:32.981709 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:32.981719 28714 leader_election.cc:302] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be4c597573ad4bc496ed19f7e09caa0a; no voters:
I20260808 07:11:32.981943 22538 external_mini_cluster.cc:1428] Setting key 5a4b1378e1987c887ba2301a836d8dfb
I20260808 07:11:32.981954 28714 leader_election.cc:288] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:32.981988 28717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:32.982111 28717 raft_consensus.cc:686] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [term 1 LEADER]: Becoming Leader. State: Replica: be4c597573ad4bc496ed19f7e09caa0a, State: Running, Role: LEADER
I20260808 07:11:32.982204 28717 consensus_queue.cc:237] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4c597573ad4bc496ed19f7e09caa0a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41845 } }
I20260808 07:11:32.982246 28714 sys_catalog.cc:564] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:32.982540 28717 sys_catalog.cc:455] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader be4c597573ad4bc496ed19f7e09caa0a. Latest consensus state: current_term: 1 leader_uuid: "be4c597573ad4bc496ed19f7e09caa0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4c597573ad4bc496ed19f7e09caa0a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41845 } } }
I20260808 07:11:32.982622 28717 sys_catalog.cc:458] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.982549 28718 sys_catalog.cc:455] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be4c597573ad4bc496ed19f7e09caa0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4c597573ad4bc496ed19f7e09caa0a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41845 } } }
I20260808 07:11:32.982801 28718 sys_catalog.cc:458] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.983093 28723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:32.983664 28723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:32.986135 28723 catalog_manager.cc:1347] Generated new cluster ID: d586c61cc5f24efeaa850a16f1414e8b
I20260808 07:11:32.986188 28723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:32.989001 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.982633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.22.2.190:41845)
0808 07:11:32.982934 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.982945 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.983244 (+ 299us) server_negotiation.cc:407] Connection header received
0808 07:11:32.983386 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.983393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.983476 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.983692 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.984279 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.985623 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.986677 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.987317 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.987448 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.988218 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.988222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.988223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.988440 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.988519 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.988536 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.988610 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.988680 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.988765 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260808 07:11:32.989961 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:11:33.006481 28723 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:33.007054 28723 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:33.011940 28723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be4c597573ad4bc496ed19f7e09caa0a: Generated new TSK 0
I20260808 07:11:33.012136 28723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:11:33.118026 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:33.021237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.22.2.190:41845)
0808 07:11:33.021297 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:33.021299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:33.021428 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:11:33.021510 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:33.021511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:33.021526 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:33.021602 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:11:33.021834 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:33.022658 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:33.023481 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:33.023859 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:33.023933 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:33.024175 (+ 242us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:11:33.117735 (+ 93560us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:11:33.117944 (+ 209us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47118: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 07:11:33.118117 28725 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47118: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 07:11:33.118264 28744 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:33.021110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41845 (local address 127.0.0.1:47118)
0808 07:11:33.021357 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:33.021368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:33.021426 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:33.021685 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:33.021686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:33.021689 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:11:33.021756 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:33.021758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:33.022775 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:33.022776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:33.023387 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:33.023389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:33.023972 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:33.023973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:33.024042 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:33.024048 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:11:33.117929 (+ 93881us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:11:33.118075 (+ 146us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.2.190:41845: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":137,"threads_started":1}
I20260808 07:11:33.118775 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 28636
2026-08-08T07:11:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (304 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (98 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:33 dist-test-slave-6k22 krb5kdc[28757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:33 dist-test-slave-6k22 krb5kdc[28757](info): set up 2 sockets
Aug 08 07:11:33 dist-test-slave-6k22 krb5kdc[28757](info): commencing operation
krb5kdc: starting...
W20260808 07:11:34.422289 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:34 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:34.543036 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41809
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:34.625339 28773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:34.625571 28773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:34.625612 28773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:34.626863 28773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:34.626935 28773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:34.626961 28773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:34.626992 28773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:34.627017 28773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:34.628523 28773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:34.628804 28773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:34.629022 28773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:34.633934 28782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:34.633946 28778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:34.633932 28780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:34.634166 28773 server_base.cc:1034] running on GCE node
I20260808 07:11:34.634402 28773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:34.634719 28773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:34.635880 28773 hybrid_clock.cc:648] HybridClock initialized: now 1786173094635884 us; error 47 us; skew 500 ppm
Aug 08 07:11:34 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:34.643311 28773 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:34.643821 28773 webserver.cc:460] Webserver started at http://127.22.2.190:38807/ using document root <none> and password file <none>
I20260808 07:11:34.644052 28773 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:34.644119 28773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:34.644270 28773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:34.645358 28773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "88a62da381ee42efbff9f47480415370"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-6k22"
server_key: "bb8567c730a12ec362b01d05af9cab48"
server_key_iv: "645ac550a1cb4f794aaf031481f38d8f"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.645732 28773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "88a62da381ee42efbff9f47480415370"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-6k22"
server_key: "bb8567c730a12ec362b01d05af9cab48"
server_key_iv: "645ac550a1cb4f794aaf031481f38d8f"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.647078 28773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:34.647852 28789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.648031 28773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:34.648140 28773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "88a62da381ee42efbff9f47480415370"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-6k22"
server_key: "bb8567c730a12ec362b01d05af9cab48"
server_key_iv: "645ac550a1cb4f794aaf031481f38d8f"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.648231 28773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:34.672896 28773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:34.675163 28773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:34.675275 28773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:34.679512 28773 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41809
I20260808 07:11:34.679584 28845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41809 every 8 connection(s)
I20260808 07:11:34.679946 28773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:34.680855 28846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:34.683104 28846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370: Bootstrap starting.
I20260808 07:11:34.683667 28846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:34.683926 28846 log.cc:824] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:34.684556 28846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370: No bootstrap required, opened a new log
I20260808 07:11:34.686117 28846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a62da381ee42efbff9f47480415370" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41809 } }
I20260808 07:11:34.686275 28846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:34.686298 28846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a62da381ee42efbff9f47480415370, State: Initialized, Role: FOLLOWER
I20260808 07:11:34.686400 28846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a62da381ee42efbff9f47480415370" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41809 } }
I20260808 07:11:34.686448 28846 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:34.686468 28846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:34.686501 28846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:34.686982 28846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a62da381ee42efbff9f47480415370" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41809 } }
I20260808 07:11:34.687089 28846 leader_election.cc:302] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88a62da381ee42efbff9f47480415370; no voters:
I20260808 07:11:34.687232 28846 leader_election.cc:288] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:34.687314 28850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:34.687493 28846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:34.687702 28850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [term 1 LEADER]: Becoming Leader. State: Replica: 88a62da381ee42efbff9f47480415370, State: Running, Role: LEADER
I20260808 07:11:34.687765 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 28773
I20260808 07:11:34.687887 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:34.687959 28850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a62da381ee42efbff9f47480415370" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41809 } }
I20260808 07:11:34.688097 22538 external_mini_cluster.cc:1428] Setting key 91af4ded1a8b04e9489a372f85b68162
I20260808 07:11:34.688441 28852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88a62da381ee42efbff9f47480415370. Latest consensus state: current_term: 1 leader_uuid: "88a62da381ee42efbff9f47480415370" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a62da381ee42efbff9f47480415370" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41809 } } }
I20260808 07:11:34.688526 28852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:34.688863 28851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88a62da381ee42efbff9f47480415370" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88a62da381ee42efbff9f47480415370" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41809 } } }
I20260808 07:11:34.688941 28851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:34.689137 28861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:34.689888 28861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:34.691949 28861 catalog_manager.cc:1347] Generated new cluster ID: f2e7594f9d324071be8377b0c5ce98ef
I20260808 07:11:34.692005 28861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:11:34 dist-test-slave-6k22 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:34.698603 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:34.688732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45234 (local address 127.22.2.190:41809)
0808 07:11:34.689136 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:34.689146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:34.689227 (+ 81us) server_negotiation.cc:407] Connection header received
0808 07:11:34.689359 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:34.689366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:34.689423 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:34.689554 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:34.690263 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.691515 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.692440 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.692857 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.692966 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:34.695331 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:34.695334 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:34.695335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:34.695346 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:34.697043 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:34.697415 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:34.697418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:34.697419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:34.697517 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:34.697751 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:34.697752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:34.697753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:34.697992 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:34.698166 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:34.698240 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:11:34.698403 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":100,"threads_started":1}
I20260808 07:11:34.701262 28861 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:34.701937 28861 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:34.709833 28861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88a62da381ee42efbff9f47480415370: Generated new TSK 0
I20260808 07:11:34.710006 28861 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:34.742820 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41809
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:34.825656 28876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:34.825857 28876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:34.825881 28876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:34.826958 28876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:34.826990 28876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:34.827015 28876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:34.828588 28876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:34.828785 28876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:34.828960 28876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:34.834471 28882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:34.834543 28885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:34.834717 28876 server_base.cc:1034] running on GCE node
W20260808 07:11:34.834753 28883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:34.834997 28876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:34.835242 28876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:34.836447 28876 hybrid_clock.cc:648] HybridClock initialized: now 1786173094836443 us; error 41 us; skew 500 ppm
Aug 08 07:11:34 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:34.839454 28876 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:34.840039 28876 webserver.cc:460] Webserver started at http://127.22.2.129:40873/ using document root <none> and password file <none>
I20260808 07:11:34.840273 28876 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:34.840318 28876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:34.840415 28876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:34.841502 28876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "5534021a8f9845259089d76012cdf653"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-6k22"
server_key: "671dc23ad4c04f8bf0467ce80b55e51c"
server_key_iv: "1c2b4cb7ab4aa186dfd003e0d1750650"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.841835 28876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "5534021a8f9845259089d76012cdf653"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-6k22"
server_key: "671dc23ad4c04f8bf0467ce80b55e51c"
server_key_iv: "1c2b4cb7ab4aa186dfd003e0d1750650"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.843217 28876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:34.844120 28898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.844322 28876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:11:34.844417 28876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "5534021a8f9845259089d76012cdf653"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-6k22"
server_key: "671dc23ad4c04f8bf0467ce80b55e51c"
server_key_iv: "1c2b4cb7ab4aa186dfd003e0d1750650"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.844496 28876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:34.868786 28876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:34.870679 28876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:34.870850 28876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:34.871084 28876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:34.871408 28876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:34.871464 28876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.871507 28876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:34.871536 28876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.877502 28876 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:35493
I20260808 07:11:34.877554 29029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:35493 every 8 connection(s)
I20260808 07:11:34.877930 28876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:11:34 dist-test-slave-6k22 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:34.887082 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:34.878673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:49461 (local address 127.22.2.190:41809)
0808 07:11:34.878719 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:34.878721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:34.878958 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:11:34.879932 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:34.879933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:34.879948 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:34.880002 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:34.880632 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.881140 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.882143 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.882352 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.882428 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:34.884515 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:34.884517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:34.884517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:34.884525 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:34.885727 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:34.886135 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:34.886136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:34.886136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:34.886198 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:34.886444 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:34.886445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:34.886446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:34.886560 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:34.886638 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:34.886905 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:11:34.886989 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:34.887256 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:34.878649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41809 (local address 127.22.2.129:49461)
0808 07:11:34.878838 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:34.878875 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:34.879820 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:34.880070 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:34.880074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:34.880344 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:34.880530 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:34.880534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.881238 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.881239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:34.882049 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:34.882051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.882448 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.882449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:34.882571 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:34.883006 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:34.883013 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:34.884420 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:34.885835 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:34.885842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:34.885844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:34.886057 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:34.886267 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:34.886268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:34.886269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:34.886375 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:34.886630 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:34.886637 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:34.886771 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 07:11:34.886883 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:11:34.887033 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260808 07:11:34.887853 29030 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41809
I20260808 07:11:34.887944 29030 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:34.888036 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 28876
I20260808 07:11:34.888154 29030 heartbeater.cc:511] Master 127.22.2.190:41809 requested a full tablet report, sending...
I20260808 07:11:34.888175 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:34.888370 22538 external_mini_cluster.cc:1428] Setting key 4d37e810feea65a1da6c56c2217fcf36
I20260808 07:11:34.888814 28806 ts_manager.cc:194] Registered new tserver with Master: 5534021a8f9845259089d76012cdf653 (127.22.2.129:35493)
I20260808 07:11:34.889701 28806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:49461
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:34.914673 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41809
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:34.998647 29036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:34.998859 29036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:34.998883 29036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:35.000087 29036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:35.000130 29036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:35.000157 29036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:35.001570 29036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:35.001770 29036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:35.001946 29036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:35.006986 29045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:35.007012 29042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:35.006986 29043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:35.007364 29036 server_base.cc:1034] running on GCE node
I20260808 07:11:35.007634 29036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:35.007879 29036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:35.009035 29036 hybrid_clock.cc:648] HybridClock initialized: now 1786173095009025 us; error 27 us; skew 500 ppm
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.011531 29036 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:35.012048 29036 webserver.cc:460] Webserver started at http://127.22.2.130:33813/ using document root <none> and password file <none>
I20260808 07:11:35.012295 29036 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:35.012364 29036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:35.012524 29036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:35.013582 29036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "58559f06061247c79b45a4822a303b66"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-6k22"
server_key: "9f3372abb9fd828441ddcd9fe81aeab1"
server_key_iv: "4d1875c06ae22b14850dc6d9996f22f0"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.013942 29036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "58559f06061247c79b45a4822a303b66"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-6k22"
server_key: "9f3372abb9fd828441ddcd9fe81aeab1"
server_key_iv: "4d1875c06ae22b14850dc6d9996f22f0"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.015296 29036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:11:35.016145 29052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.016333 29036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:11:35.016438 29036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "58559f06061247c79b45a4822a303b66"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-6k22"
server_key: "9f3372abb9fd828441ddcd9fe81aeab1"
server_key_iv: "4d1875c06ae22b14850dc6d9996f22f0"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.016520 29036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:35.028909 29036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:35.031093 29036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:35.031262 29036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:35.031505 29036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:35.031849 29036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:35.031908 29036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.031953 29036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:35.031991 29036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.038515 29036 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:40721
I20260808 07:11:35.038550 29175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:40721 every 8 connection(s)
I20260808 07:11:35.038897 29036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:35.040017 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 29036
I20260808 07:11:35.040156 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:35.040355 22538 external_mini_cluster.cc:1428] Setting key b519588193d7a8ae6bf7e7b5c230c09b
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 07:11:35.050088 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.039699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:34499 (local address 127.22.2.190:41809)
0808 07:11:35.039748 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.039750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.040089 (+ 339us) server_negotiation.cc:407] Connection header received
0808 07:11:35.041189 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.041190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.041204 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.041253 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.042122 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.042631 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.043829 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.044242 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.044334 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.046802 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:35.046804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:35.046804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:35.046813 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:35.048537 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.048995 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.048997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.048997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.049064 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.049350 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.049351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.049351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.049477 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:35.049562 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.049879 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.049989 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:35.050279 29178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.039704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41809 (local address 127.22.2.130:34499)
0808 07:11:35.039920 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.039971 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:35.041103 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:35.041331 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:35.041335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:35.041715 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:35.042003 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.042009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.042728 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.042729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.043738 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.043741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.044359 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.044360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.044510 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.045001 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:35.045011 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:35.046696 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:35.048637 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:35.048645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:35.048647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:35.048894 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:35.049140 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:35.049141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:35.049142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:35.049275 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:35.049555 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:35.049562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:35.049727 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.049842 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.050014 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":115,"threads_started":1}
I20260808 07:11:35.050853 29176 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41809
I20260808 07:11:35.050963 29176 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:35.051141 29176 heartbeater.cc:511] Master 127.22.2.190:41809 requested a full tablet report, sending...
I20260808 07:11:35.051545 28806 ts_manager.cc:194] Registered new tserver with Master: 58559f06061247c79b45a4822a303b66 (127.22.2.130:40721)
I20260808 07:11:35.052165 28806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:34499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:35.066481 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41809
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:35.149327 29181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:35.149515 29181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:35.149540 29181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:35.150676 29181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:35.150712 29181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:35.150738 29181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:35.152282 29181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:35.152462 29181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:35.152626 29181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:35.157567 29188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:35.157567 29186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:35.157677 29190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:35.157737 29181 server_base.cc:1034] running on GCE node
I20260808 07:11:35.158119 29181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:35.158377 29181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:35.159538 29181 hybrid_clock.cc:648] HybridClock initialized: now 1786173095159525 us; error 45 us; skew 500 ppm
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.162210 29181 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:35.162634 29181 webserver.cc:460] Webserver started at http://127.22.2.131:40825/ using document root <none> and password file <none>
I20260808 07:11:35.162858 29181 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:35.162925 29181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:35.163105 29181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:35.164598 29181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "d68aaff9058a4ce6bafeadf9f006f22a"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-6k22"
server_key: "a4980ba5e263353691b3a52d09d7f919"
server_key_iv: "7dd0d0bc07a2d2b0db8bacbfb8176a72"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.164958 29181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "d68aaff9058a4ce6bafeadf9f006f22a"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-6k22"
server_key: "a4980ba5e263353691b3a52d09d7f919"
server_key_iv: "7dd0d0bc07a2d2b0db8bacbfb8176a72"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.166314 29181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:11:35.167047 29197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.167249 29181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:35.167358 29181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "d68aaff9058a4ce6bafeadf9f006f22a"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-6k22"
server_key: "a4980ba5e263353691b3a52d09d7f919"
server_key_iv: "7dd0d0bc07a2d2b0db8bacbfb8176a72"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.167438 29181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:35.176000 29181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:35.177806 29181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:35.177997 29181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:35.178213 29181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:35.178519 29181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:35.178573 29181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.178613 29181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:35.178642 29181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.184697 29181 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:41513
I20260808 07:11:35.184741 29322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:41513 every 8 connection(s)
I20260808 07:11:35.185040 29181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:35.191179 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 29181
I20260808 07:11:35.191254 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:35.191481 22538 external_mini_cluster.cc:1428] Setting key 8eb2218fc8491f1cbb998f0723fdd333
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:35.195039 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.185727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:34459 (local address 127.22.2.190:41809)
0808 07:11:35.185771 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.185773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.186004 (+ 231us) server_negotiation.cc:407] Connection header received
0808 07:11:35.186961 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.186961 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.186970 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.187045 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.187891 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.188435 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.189755 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.189946 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.189997 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.192481 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:35.192483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:35.192483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:35.192491 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:35.193686 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.194089 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.194090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.194090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.194153 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.194401 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.194401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.194402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.194518 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:35.194594 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.194871 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.194952 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:11:35.195212 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.185684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41809 (local address 127.22.2.131:34459)
0808 07:11:35.185866 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.185911 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:35.186874 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:35.187120 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:35.187123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:35.187532 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:35.187810 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.187816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.188529 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.188530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.189665 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.189668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.190017 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.190018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.190119 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.190664 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:35.190672 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:35.192391 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:35.193780 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:35.193788 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:35.193789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:35.194014 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:35.194220 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:35.194220 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:35.194221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:35.194333 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:35.194585 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:35.194592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:35.194725 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.194835 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.194984 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260808 07:11:35.195657 29323 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41809
I20260808 07:11:35.195735 29323 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:35.195921 29323 heartbeater.cc:511] Master 127.22.2.190:41809 requested a full tablet report, sending...
I20260808 07:11:35.196275 28806 ts_manager.cc:194] Registered new tserver with Master: d68aaff9058a4ce6bafeadf9f006f22a (127.22.2.131:41513)
I20260808 07:11:35.196667 28806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:34459
I20260808 07:11:35.203406 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/test-admin.keytab.
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.221498 22538 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 07:11:35.223469 29331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:35.222477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.22.2.129:35493)
0808 07:11:35.222711 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.222715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.222725 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:35.222753 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.222755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.222770 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.222863 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.223374 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59772: BlockingRecv error: recv got EOF from 127.0.0.1:59772 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":70,"threads_started":1}
W20260808 07:11:35.223554 22538 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.224596 22538 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.225474 22538 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:35.233286 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.226014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.22.2.129:35493)
0808 07:11:35.226070 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.226073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.226105 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:11:35.226196 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.226197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.226211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.226281 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.226674 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.227428 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.228225 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.228538 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.228616 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.230796 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:35.230813 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:35.230813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:35.230820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:35.232286 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.232595 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.232597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.232597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.232649 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.232800 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.232801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.232801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.232927 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:35.233027 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.233080 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.233136 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 07:11:35.234968 29331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:35.234495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.22.2.129:35493)
0808 07:11:35.234544 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.234546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.234555 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:35.234604 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.234605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.234613 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.234660 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.234903 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59790: BlockingRecv error: recv got EOF from 127.0.0.1:59790 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 07:11:35.235066 22538 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.236044 22538 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.236872 22538 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:35.243146 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.237304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.22.2.129:35493)
0808 07:11:35.237338 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.237339 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.237368 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:11:35.237415 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.237416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.237422 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.237470 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.237816 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.238268 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.238906 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.239089 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.239136 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.241132 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:35.241133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:35.241134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:35.241139 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:35.242227 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.242533 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.242534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.242534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.242582 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.242748 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.242749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.242749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.242830 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:35.242898 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.242977 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.243030 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 07:11:35.244601 29331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:35.244066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59820 (local address 127.22.2.129:35493)
0808 07:11:35.244098 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.244100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.244170 (+ 70us) server_negotiation.cc:407] Connection header received
0808 07:11:35.244238 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.244239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.244248 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.244301 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.244557 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59820: BlockingRecv error: recv got EOF from 127.0.0.1:59820 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 07:11:35.244710 22538 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173068742558-22538-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.245618 22538 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.246423 22538 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:11:35.252964 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.246926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59828 (local address 127.22.2.129:35493)
0808 07:11:35.246970 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.246972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.247056 (+ 84us) server_negotiation.cc:407] Connection header received
0808 07:11:35.247133 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.247134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.247142 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.247219 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.247592 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.248133 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.248804 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.249004 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.249085 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.250948 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:35.250950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:35.250950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:35.250956 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:35.252167 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.252451 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.252452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.252452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.252498 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.252646 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.252647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.252647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.252722 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:35.252781 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.252868 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.252916 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:11:35.253582 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 28876
I20260808 07:11:35.261273 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 29036
I20260808 07:11:35.268841 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 29181
I20260808 07:11:35.276333 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 28773
2026-08-08T07:11:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1966 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[29345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[29345](info): set up 2 sockets
Aug 08 07:11:35 dist-test-slave-6k22 krb5kdc[29345](info): commencing operation
krb5kdc: starting...
W20260808 07:11:36.373342 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:36 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.190.keytab.
Entry for principal oryx/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.190.keytab.
I20260808 07:11:36.495345 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41569
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.190.keytab
--principal=oryx/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:36.590974 29360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:36.591197 29360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:36.591236 29360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:36.592388 29360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:36.592456 29360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:36.592481 29360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:36.592514 29360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:36.592541 29360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:36.593961 29360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.190.keytab
--principal=oryx/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:36.594225 29360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:36.594446 29360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:36.599294 29360 server_base.cc:1034] running on GCE node
W20260808 07:11:36.599294 29370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:36.599346 29367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:36.599323 29368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:36.599781 29360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:36.600082 29360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:36.601238 29360 hybrid_clock.cc:648] HybridClock initialized: now 1786173096601215 us; error 31 us; skew 500 ppm
Aug 08 07:11:36 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:36.603605 29360 init.cc:377] Logged in from keytab as oryx/127.22.2.190@KRBTEST.COM (short username oryx)
I20260808 07:11:36.604063 29360 webserver.cc:460] Webserver started at http://127.22.2.190:44997/ using document root <none> and password file <none>
I20260808 07:11:36.604285 29360 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:36.604334 29360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:36.604480 29360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:36.605526 29360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "9ed2c8af33b24d7a9ae45e99e07787c5"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "b242b5d98d45025dc9ec134b447a66bf"
server_key_iv: "b65f70bc19b5ae8067fd0ea87fa67533"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.605901 29360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "9ed2c8af33b24d7a9ae45e99e07787c5"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "b242b5d98d45025dc9ec134b447a66bf"
server_key_iv: "b65f70bc19b5ae8067fd0ea87fa67533"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.607218 29360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:11:36.608009 29377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:36.608181 29360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:36.608285 29360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "9ed2c8af33b24d7a9ae45e99e07787c5"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "b242b5d98d45025dc9ec134b447a66bf"
server_key_iv: "b65f70bc19b5ae8067fd0ea87fa67533"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.608366 29360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:36.614907 29360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:36.616641 29360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:36.616794 29360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:36.621037 29360 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41569
I20260808 07:11:36.621068 29436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41569 every 8 connection(s)
I20260808 07:11:36.621383 29360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:36.622318 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:36.624539 29437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: Bootstrap starting.
I20260808 07:11:36.625118 29437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:36.625408 29437 log.cc:824] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:36.626165 29437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: No bootstrap required, opened a new log
I20260808 07:11:36.627827 29437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } }
I20260808 07:11:36.627955 29437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:36.627977 29437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ed2c8af33b24d7a9ae45e99e07787c5, State: Initialized, Role: FOLLOWER
I20260808 07:11:36.628144 29437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } }
I20260808 07:11:36.628232 29437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:36.628278 29437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:36.628325 29437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:36.629079 29437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } }
I20260808 07:11:36.629197 29437 leader_election.cc:302] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ed2c8af33b24d7a9ae45e99e07787c5; no voters:
I20260808 07:11:36.629371 29437 leader_election.cc:288] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:36.629411 29442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:36.629587 29442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 1 LEADER]: Becoming Leader. State: Replica: 9ed2c8af33b24d7a9ae45e99e07787c5, State: Running, Role: LEADER
I20260808 07:11:36.629617 29437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:36.629673 29442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } }
I20260808 07:11:36.630082 29443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } }
I20260808 07:11:36.630098 29444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ed2c8af33b24d7a9ae45e99e07787c5. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } }
I20260808 07:11:36.630265 29443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:36.630278 29444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:36.630399 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 29360
I20260808 07:11:36.630522 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:36.630606 29451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:36.630749 22538 external_mini_cluster.cc:1428] Setting key 98689ff3a76f2877e3c639616e504c95
I20260808 07:11:36.631119 29451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:36.632606 29451 catalog_manager.cc:1347] Generated new cluster ID: ddfba8ff318b41969f8a285d353b4125
I20260808 07:11:36.632659 29451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:11:36 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:36.643536 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:36.631349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.22.2.190:41569)
0808 07:11:36.631527 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:36.631536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:36.631654 (+ 118us) server_negotiation.cc:407] Connection header received
0808 07:11:36.631819 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:36.631826 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:36.631885 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:36.632822 (+ 937us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:36.632921 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:36.634268 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:36.635371 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:36.636029 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:36.636445 (+ 416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:36.639217 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:36.639221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:36.639221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:36.639233 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:36.641400 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:36.641829 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:36.641832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:36.641832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:36.641950 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:36.642166 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:36.642166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:36.642167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:36.642477 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:36.643092 (+ 615us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:36.643165 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 07:11:36.643303 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260808 07:11:36.656531 29451 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:36.657068 29451 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:36.660823 29451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: Generated new TSK 0
I20260808 07:11:36.661003 29451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
I20260808 07:11:36.696498 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41569
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:36.797240 29468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:36.797442 29468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:36.797467 29468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:36.798631 29468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:36.798672 29468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:36.798700 29468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:36.800165 29468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:36.800392 29468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:36.800611 29468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:36.805775 29476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:36.805776 29473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:36.805909 29468 server_base.cc:1034] running on GCE node
W20260808 07:11:36.805946 29474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:36.806198 29468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:36.806393 29468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:36.807533 29468 hybrid_clock.cc:648] HybridClock initialized: now 1786173096807512 us; error 36 us; skew 500 ppm
Aug 08 07:11:36 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:36.809919 29468 init.cc:377] Logged in from keytab as oryx/127.22.2.129@KRBTEST.COM (short username oryx)
I20260808 07:11:36.810390 29468 webserver.cc:460] Webserver started at http://127.22.2.129:46369/ using document root <none> and password file <none>
I20260808 07:11:36.810631 29468 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:36.810696 29468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:36.810892 29468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:36.812076 29468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "97880488f60b4581b7f9fb132157599b"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "c0f2f2e7edbe0de2b77123846118c58b"
server_key_iv: "be231a65bbe32ef8651f37224c62d5b6"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.812441 29468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "97880488f60b4581b7f9fb132157599b"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "c0f2f2e7edbe0de2b77123846118c58b"
server_key_iv: "be231a65bbe32ef8651f37224c62d5b6"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.813727 29468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:36.814497 29483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:36.814677 29468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:36.814776 29468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "97880488f60b4581b7f9fb132157599b"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "c0f2f2e7edbe0de2b77123846118c58b"
server_key_iv: "be231a65bbe32ef8651f37224c62d5b6"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.814865 29468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:36.846652 29468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:36.848541 29468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:36.848706 29468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:36.848954 29468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:36.849282 29468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:36.849339 29468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:36.849382 29468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:36.849412 29468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:36.855290 29468 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:46071
I20260808 07:11:36.855357 29611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:46071 every 8 connection(s)
I20260808 07:11:36.855666 29468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:11:36 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:36.862998 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 29468
I20260808 07:11:36.863112 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:36.863380 22538 external_mini_cluster.cc:1428] Setting key ead8d8cdc79427c89d5b09ae4b32efa1
I20260808 07:11:36.865301 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:36.856377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:37517 (local address 127.22.2.190:41569)
0808 07:11:36.856450 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:36.856453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:36.856672 (+ 219us) server_negotiation.cc:407] Connection header received
0808 07:11:36.857133 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:36.857134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:36.857145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:36.857199 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:36.858107 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:36.858749 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:36.859728 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:36.859953 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:36.860030 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:36.862160 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:36.862161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:36.862161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:36.862169 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:36.863932 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:36.864352 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:36.864353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:36.864354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:36.864420 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:36.864666 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:36.864667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:36.864667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:36.864782 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:36.864861 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:36.865127 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 07:11:36.865210 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:11:36.865499 29614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:36.856347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:37517)
0808 07:11:36.856550 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:36.856586 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:36.857050 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:36.857246 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:36.857249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:36.857790 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:36.857984 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:36.857989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:36.858852 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:36.858853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:36.859635 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:36.859637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:36.860038 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:36.860039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:36.860132 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:36.860597 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:36.860604 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:36.862063 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:36.864025 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:36.864033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:36.864035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:36.864252 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:36.864489 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:36.864490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:36.864491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:36.864596 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:36.864863 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:36.864870 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:36.864997 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 07:11:36.865102 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:11:36.865252 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260808 07:11:36.865924 29612 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41569
I20260808 07:11:36.866014 29612 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:36.866194 29612 heartbeater.cc:511] Master 127.22.2.190:41569 requested a full tablet report, sending...
I20260808 07:11:36.866740 29397 ts_manager.cc:194] Registered new tserver with Master: 97880488f60b4581b7f9fb132157599b (127.22.2.129:46071)
I20260808 07:11:36.867408 29397 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:37517
WARNING: no policy specified for oryx/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
I20260808 07:11:36.889848 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41569
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:36.973351 29618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:36.973554 29618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:36.973577 29618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:36.974758 29618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:36.974799 29618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:36.974825 29618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:36.976235 29618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:36.976423 29618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:36.976589 29618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:36.981743 29623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:36.981753 29624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:36.981789 29626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:36.982053 29618 server_base.cc:1034] running on GCE node
I20260808 07:11:36.982295 29618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:36.982532 29618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:36.983708 29618 hybrid_clock.cc:648] HybridClock initialized: now 1786173096983664 us; error 32 us; skew 500 ppm
Aug 08 07:11:36 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:36.986173 29618 init.cc:377] Logged in from keytab as oryx/127.22.2.130@KRBTEST.COM (short username oryx)
I20260808 07:11:36.986644 29618 webserver.cc:460] Webserver started at http://127.22.2.130:34583/ using document root <none> and password file <none>
I20260808 07:11:36.986891 29618 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:36.986961 29618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:36.987092 29618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:36.988171 29618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "0296fb38106f45de9a3b43df9b814368"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "9e9e48b6065f1964b9cef851493c4b8f"
server_key_iv: "3ff00ab2e78b516e6f936005b4bcb1db"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.988528 29618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "0296fb38106f45de9a3b43df9b814368"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "9e9e48b6065f1964b9cef851493c4b8f"
server_key_iv: "3ff00ab2e78b516e6f936005b4bcb1db"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.989831 29618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:36.990525 29634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:36.990698 29618 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:36.990805 29618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "0296fb38106f45de9a3b43df9b814368"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "9e9e48b6065f1964b9cef851493c4b8f"
server_key_iv: "3ff00ab2e78b516e6f936005b4bcb1db"
server_key_version: "encryptionkey@0"
I20260808 07:11:36.990878 29618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.001860 29618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:37.004192 29618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:37.004357 29618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:37.004618 29618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:37.004940 29618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:37.004976 29618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.004998 29618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:37.005012 29618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.011297 29618 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:38215
I20260808 07:11:37.011341 29757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:38215 every 8 connection(s)
I20260808 07:11:37.011855 29618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:37.015158 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 29618
I20260808 07:11:37.015286 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:37.015489 22538 external_mini_cluster.cc:1428] Setting key b4b4629c2c75334e93e4d27b631661a5
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:37.022141 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.012594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:37885)
0808 07:11:37.012739 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.012788 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.013268 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.013467 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.013470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.014035 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.014227 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.014232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.014871 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.014872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.016153 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.016156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.016546 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.016547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.016687 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.017268 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.017276 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.018865 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.020696 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.020703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.020704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.021020 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.021236 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.021237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.021237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.021364 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.021669 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.021673 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.021783 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.021866 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.021983 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":88,"threads_started":1}
I20260808 07:11:37.022151 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.012597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:37885 (local address 127.22.2.190:41569)
0808 07:11:37.012642 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.012644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.012876 (+ 232us) server_negotiation.cc:407] Connection header received
0808 07:11:37.013351 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.013352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.013362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.013415 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.014305 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.014782 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.016233 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.016457 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.016512 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.018961 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.018963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.018963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.018971 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.020611 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.021100 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.021102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.021102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.021166 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.021445 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.021446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.021446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.021596 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.021675 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.021916 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.022011 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:37.022636 29758 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41569
I20260808 07:11:37.022749 29758 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:37.022943 29758 heartbeater.cc:511] Master 127.22.2.190:41569 requested a full tablet report, sending...
I20260808 07:11:37.023396 29397 ts_manager.cc:194] Registered new tserver with Master: 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215)
I20260808 07:11:37.023820 29397 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:37885
WARNING: no policy specified for oryx/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
I20260808 07:11:37.042254 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41569
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:37.124817 29766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:37.125039 29766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:37.125079 29766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:37.126271 29766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:37.126330 29766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:37.126375 29766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:37.127965 29766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:37.128226 29766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:37.128448 29766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:37.133793 29772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.133769 29771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.133755 29774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.133808 29766 server_base.cc:1034] running on GCE node
I20260808 07:11:37.134212 29766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:37.134408 29766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:37.135597 29766 hybrid_clock.cc:648] HybridClock initialized: now 1786173097135541 us; error 27 us; skew 500 ppm
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:37.138054 29766 init.cc:377] Logged in from keytab as oryx/127.22.2.131@KRBTEST.COM (short username oryx)
I20260808 07:11:37.138517 29766 webserver.cc:460] Webserver started at http://127.22.2.131:34033/ using document root <none> and password file <none>
I20260808 07:11:37.138715 29766 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.138754 29766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.138842 29766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:37.139892 29766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "d26a25f00305411abd569f0a54ea6cbf"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "5d232936f53c93e6844ca132d54e3f0e"
server_key_iv: "db6c2e12e5671bf6aef4ad25ace1090f"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.140203 29766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "d26a25f00305411abd569f0a54ea6cbf"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "5d232936f53c93e6844ca132d54e3f0e"
server_key_iv: "db6c2e12e5671bf6aef4ad25ace1090f"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.141476 29766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:37.142163 29781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.142350 29766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:37.142442 29766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "d26a25f00305411abd569f0a54ea6cbf"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "5d232936f53c93e6844ca132d54e3f0e"
server_key_iv: "db6c2e12e5671bf6aef4ad25ace1090f"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.142515 29766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.162829 29766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:37.164904 29766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:37.165052 29766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:37.165298 29766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:37.165618 29766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:37.165678 29766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.165721 29766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:37.165750 29766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.171609 29766 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:41063
I20260808 07:11:37.171667 29914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:41063 every 8 connection(s)
I20260808 07:11:37.171960 29766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:37.177330 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 29766
I20260808 07:11:37.177443 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:37.177634 22538 external_mini_cluster.cc:1428] Setting key 7709031cdf16b9ccae668b18ff641524
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:37.181020 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.172656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:46941 (local address 127.22.2.190:41569)
0808 07:11:37.172716 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.172719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.172956 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:11:37.173412 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.173414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.173424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.173478 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.174462 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.174952 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.175974 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.176157 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.176213 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.178447 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.178448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.178449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.178456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.179769 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.180129 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.180129 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.180130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.180177 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.180386 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.180386 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.180386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.180484 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.180540 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.180889 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.180958 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:11:37.181025 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.172646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:46941)
0808 07:11:37.172828 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.172865 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.173324 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.173554 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.173557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.174194 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.174381 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.174385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.175028 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.175029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.175897 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.175899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.176229 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.176229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.176324 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.176773 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.176781 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.178328 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.179837 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.179843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.179844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.180066 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.180234 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.180235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.180235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.180332 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.180548 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.180553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.180660 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.180752 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.180870 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260808 07:11:37.181581 29915 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41569
I20260808 07:11:37.181689 29915 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:37.181867 29915 heartbeater.cc:511] Master 127.22.2.190:41569 requested a full tablet report, sending...
I20260808 07:11:37.182231 29397 ts_manager.cc:194] Registered new tserver with Master: d26a25f00305411abd569f0a54ea6cbf (127.22.2.131:41063)
I20260808 07:11:37.182591 29397 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:46941
I20260808 07:11:37.189373 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.2.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.189.keytab.
Entry for principal oryx/127.22.2.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.189.keytab.
I20260808 07:11:37.218123 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:37763
--webserver_interface=127.22.2.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.189.keytab
--principal=oryx/127.22.2.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41569,127.22.2.189:37763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:37.304767 29920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:37.304979 29920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:37.305001 29920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:37.306056 29920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:37.306089 29920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:37.306103 29920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:37.306115 29920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:37.306128 29920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:37.307488 29920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41569,127.22.2.189:37763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.189.keytab
--principal=oryx/127.22.2.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:37763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.2.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
W20260808 07:11:37.307610 29920 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.2.190:41569,127.22.2.189:37763'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 07:11:37.307825 29920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:37.308058 29920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:37.312958 29925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.313004 29926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.313041 29928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.313078 29920 server_base.cc:1034] running on GCE node
I20260808 07:11:37.313470 29920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:37.313717 29920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:37.314859 29920 hybrid_clock.cc:648] HybridClock initialized: now 1786173097314837 us; error 40 us; skew 500 ppm
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:37.317493 29920 init.cc:377] Logged in from keytab as oryx/127.22.2.189@KRBTEST.COM (short username oryx)
I20260808 07:11:37.317975 29920 webserver.cc:460] Webserver started at http://127.22.2.189:37795/ using document root <none> and password file <none>
I20260808 07:11:37.318243 29920 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.318321 29920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.318490 29920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:37.319532 29920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data/instance:
uuid: "9f3f91336b2a4abf9dbdbc4652149177"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "8f761ca31e7913e666fbca7d9c3f1c14"
server_key_iv: "b4151bd1c1cff8961f95c5405852b323"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.319975 29920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal/instance:
uuid: "9f3f91336b2a4abf9dbdbc4652149177"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "8f761ca31e7913e666fbca7d9c3f1c14"
server_key_iv: "b4151bd1c1cff8961f95c5405852b323"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.321242 29920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:37.321941 29935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.322111 29920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:37.322214 29920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
uuid: "9f3f91336b2a4abf9dbdbc4652149177"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "8f761ca31e7913e666fbca7d9c3f1c14"
server_key_iv: "b4151bd1c1cff8961f95c5405852b323"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.322295 29920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.330655 29920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:37.332551 29920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:37.332726 29920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:37.336956 29920 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:37763
I20260808 07:11:37.337005 29992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:37763 every 8 connection(s)
I20260808 07:11:37.337355 29920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
I20260808 07:11:37.338256 29993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:37.339737 29993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:37.343791 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 29920
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.189@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:37.349102 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.340375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.22.2.190:41569)
0808 07:11:37.340442 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.340445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.340620 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:11:37.341139 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.341141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.341152 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.341203 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.341790 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.342264 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.343258 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.343459 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.343600 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.346229 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.346230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.346231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.346237 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.347699 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.348093 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.348094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.348095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.348160 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.348427 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.348428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.348428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.348580 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.348654 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.348928 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.349011 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:11:37.349272 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.340352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:44158)
0808 07:11:37.340502 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.340542 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.341029 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.341264 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.341267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.341511 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.341674 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.341677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.342367 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.342368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.343149 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.343151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.343591 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.343592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.343736 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.344241 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.344251 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.346149 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.347779 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.347793 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.347795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.348012 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.348235 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.348236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.348236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.348350 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.348658 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.348665 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.348799 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.348907 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.349048 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 07:11:37.349843 29993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:37.353183 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.344306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.22.2.189:37763)
0808 07:11:37.344446 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.344450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.344544 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:11:37.344568 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.344569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.344586 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.344649 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.345494 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.346513 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.347454 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.347823 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.347913 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.350440 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.350447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.350447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.350453 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.351987 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.352341 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.352343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.352344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.352409 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.352611 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.352612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.352612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.352730 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.352910 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.352954 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.353083 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.189@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 07:11:37.356936 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.350093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.0.0.1:48256)
0808 07:11:37.350136 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.350145 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.350181 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.350408 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.350409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.350557 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.350598 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.350600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.351510 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.351511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.352060 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.352062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.352437 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.352455 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.352530 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.352921 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.352926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.354244 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.356122 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.356123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.356123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.356277 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.356479 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.356480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.356480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.356527 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.356728 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.356728 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.356761 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.356812 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.356881 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 07:11:37.357087 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.350132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.22.2.189:37763)
0808 07:11:37.350274 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.350276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.350283 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:37.350316 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.350316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.350324 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.350380 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.350670 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.351429 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.352142 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.352363 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.352432 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.354321 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.354321 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.354322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.354327 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.356053 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.356353 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.356354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.356354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.356421 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.356580 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.356580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.356581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.356669 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.356716 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.356845 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.356926 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260808 07:11:37.358255 29993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Bootstrap starting.
I20260808 07:11:37.358871 29993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:37.359144 29993 log.cc:824] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:37.359848 29993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: No bootstrap required, opened a new log
I20260808 07:11:37.361186 29993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } }
I20260808 07:11:37.361299 29993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:37.361363 29993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3f91336b2a4abf9dbdbc4652149177, State: Initialized, Role: FOLLOWER
I20260808 07:11:37.361485 29993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } }
I20260808 07:11:37.361706 30001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } } }
I20260808 07:11:37.361748 29993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:37.361790 30001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:11:37.363502 30015 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:37.363616 30015 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:37.365352 29920 master_runner.cc:428] Detected that this master 9f3f91336b2a4abf9dbdbc4652149177 is joining an existing cluster
I20260808 07:11:37.365410 29920 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.2.189:37763
I20260808 07:11:37.373292 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.366433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.0.0.1:48260)
0808 07:11:37.366608 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.366617 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.366674 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.366809 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.366810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.367079 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.367161 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.367163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.368073 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.368074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.368949 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.368952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.369332 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.369333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.369433 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.369875 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.369879 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.370273 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.372141 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.372142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.372142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.372386 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.372597 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.372597 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.372597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.372649 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.372951 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.372951 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.372984 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.373067 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.373165 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260808 07:11:37.373755 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.366590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.22.2.189:37763)
0808 07:11:37.366621 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.366637 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.366660 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:11:37.366722 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.366723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.366732 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.366786 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.367256 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.367992 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.369044 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.369237 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.369319 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.370361 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.370362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.370363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.370371 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.372057 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.372445 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.372446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.372447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.372498 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.372740 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.372741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.372741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.372869 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.372930 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.373504 (+ 574us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.373639 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:11:37.374534 30022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.366398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:44162)
0808 07:11:37.366551 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.366562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.366642 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.366833 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.366834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.367035 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.367124 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.367125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.369063 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.369064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.369958 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.369960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.370420 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.370420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.370484 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.371011 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.371016 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.371468 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.373426 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.373427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.373428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.373681 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.373929 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.373930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.373930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.373996 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.374302 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.374303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.374353 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.374400 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.374470 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":19,"thread_start_us":99,"threads_started":1}
I20260808 07:11:37.374620 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.366428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.22.2.190:41569)
0808 07:11:37.366469 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.366472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.366615 (+ 143us) server_negotiation.cc:407] Connection header received
0808 07:11:37.366726 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.366727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.366739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.366771 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.367218 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.367994 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.370049 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.370352 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.370420 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.371587 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.371588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.371589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.371596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.373334 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.373779 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.373780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.373780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.373846 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.374080 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.374082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.374082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.374214 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.374258 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.374442 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.374531 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 07:11:37.384079 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.377018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.22.2.189:37763)
0808 07:11:37.377110 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.377111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.377173 (+ 62us) server_negotiation.cc:407] Connection header received
0808 07:11:37.377270 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.377270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.377277 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.377317 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.377794 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.378221 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.379294 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.379457 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.379506 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.381561 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.381563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.381563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.381570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.382851 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.383239 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.383240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.383241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.383293 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.383491 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.383491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.383492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.383643 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.383723 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.383922 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.383990 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 07:11:37.384101 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.376943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.0.0.1:48268)
0808 07:11:37.377082 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.377122 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.377199 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.377337 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.377340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.377664 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.377720 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.377722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.378278 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.378280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.379212 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.379215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.379526 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.379527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.379638 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.380020 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.380026 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.381440 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.382935 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.382942 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.382944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.383146 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.383358 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.383359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.383360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.383425 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.383717 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.383725 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.383774 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.383870 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.383993 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260808 07:11:37.384564 29397 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.2.189:37763: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177"
member_type: NON_VOTER
last_known_addr {
host: "127.22.2.189"
port: 37763
}
attrs {
promote: true
}
}
dest_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5"
cas_config_opid_index: -1
I20260808 07:11:37.384929 29397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } }
I20260808 07:11:37.385399 30027 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } }
I20260808 07:11:37.385699 29964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:37.386351 30027 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.2.189:37763
I20260808 07:11:37.386631 29920 master.cc:537] Master@127.22.2.189:37763 shutting down...
I20260808 07:11:37.386641 29964 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Refusing update from remote peer 9ed2c8af33b24d7a9ae45e99e07787c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 07:11:37.386623 30025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } }
I20260808 07:11:37.386724 30025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:37.387063 30025 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:11:37.387877 29380 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 07:11:37.388288 29380 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 -> Peer 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189:37763): Couldn't send request to peer 9f3f91336b2a4abf9dbdbc4652149177. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:11:37.389394 29920 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:37.389482 29920 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:37.389534 29920 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: stopping tablet replica
I20260808 07:11:37.402170 29920 master.cc:552] Master@127.22.2.189:37763 shutdown complete.
I20260808 07:11:37.402246 29920 master_runner.cc:305] Clearing existing system tablet
I20260808 07:11:37.402375 29920 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.402438 29920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.403061 29920 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.403582 30032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.403754 29920 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.403831 29920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:37.403862 29920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
uuid: "9f3f91336b2a4abf9dbdbc4652149177"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "8f761ca31e7913e666fbca7d9c3f1c14"
server_key_iv: "b4151bd1c1cff8961f95c5405852b323"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.404772 29920 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:37.406453 29920 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:11:37.406535 29920 log.cc:1196] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 07:11:37.406826 29920 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Deleting consensus metadata
I20260808 07:11:37.406968 29920 master_runner.cc:315] Copying system tablet from 127.22.2.190:41569
I20260808 07:11:37.407714 29920 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: tablet copy: Beginning tablet copy session from remote peer at address 127.22.2.190:41569
I20260808 07:11:37.414899 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.408069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44172 (local address 127.22.2.190:41569)
0808 07:11:37.408115 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.408117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.408257 (+ 140us) server_negotiation.cc:407] Connection header received
0808 07:11:37.408303 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.408304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.408313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.408369 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.409092 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.409577 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.410758 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.410985 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.411084 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.412345 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.412346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.412347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.412353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.413775 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.414182 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.414182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.414183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.414243 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.414452 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.414452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.414452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.414561 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.414612 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.414779 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.414837 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:37.414984 30038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.408015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:44172)
0808 07:11:37.408156 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.408180 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.408243 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.408395 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.408396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.408632 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.409006 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.409009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.409661 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.409662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.410676 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.410679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.411083 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.411084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.411170 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.411818 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.411824 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.412261 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.413858 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.413860 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.413860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.414102 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.414310 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.414311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.414311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.414387 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.414626 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.414627 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.414677 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.414756 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.414867 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260808 07:11:37.415483 29420 tablet_copy_service.cc:140] P 9ed2c8af33b24d7a9ae45e99e07787c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9f3f91336b2a4abf9dbdbc4652149177 ({username='oryx', principal='oryx/127.22.2.189@KRBTEST.COM'} at 127.0.0.1:44172)
I20260808 07:11:37.415596 29420 tablet_copy_service.cc:161] P 9ed2c8af33b24d7a9ae45e99e07787c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9f3f91336b2a4abf9dbdbc4652149177 at {username='oryx', principal='oryx/127.22.2.189@KRBTEST.COM'} at 127.0.0.1:44172: session id = 9f3f91336b2a4abf9dbdbc4652149177-00000000000000000000000000000000
I20260808 07:11:37.416218 29420 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:11:37.416836 29920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:11:37.418179 29920 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: tablet copy: Starting download of 0 data blocks...
I20260808 07:11:37.418354 29920 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: tablet copy: Starting download of 1 WAL segments...
I20260808 07:11:37.419289 29920 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:11:37.421231 29420 tablet_copy_service.cc:342] P 9ed2c8af33b24d7a9ae45e99e07787c5: Request end of tablet copy session 9f3f91336b2a4abf9dbdbc4652149177-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.2.189@KRBTEST.COM'} at 127.0.0.1:44172
I20260808 07:11:37.421298 29420 tablet_copy_service.cc:433] P 9ed2c8af33b24d7a9ae45e99e07787c5: ending tablet copy session 9f3f91336b2a4abf9dbdbc4652149177-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9f3f91336b2a4abf9dbdbc4652149177
I20260808 07:11:37.423991 29920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:37.425155 30039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.425225 30042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.425305 30040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.425531 29920 server_base.cc:1034] running on GCE node
I20260808 07:11:37.425640 29920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:37.425787 29920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:37.426910 29920 hybrid_clock.cc:648] HybridClock initialized: now 1786173097426898 us; error 27 us; skew 500 ppm
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:37.428274 29920 init.cc:377] Logged in from keytab as oryx/127.22.2.189@KRBTEST.COM (short username oryx)
I20260808 07:11:37.428555 29920 webserver.cc:460] Webserver started at http://127.22.2.189:36105/ using document root <none> and password file <none>
I20260808 07:11:37.428692 29920 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.428735 29920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.429410 29920 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:37.429992 30050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.430111 29920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:37.430161 29920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
uuid: "9f3f91336b2a4abf9dbdbc4652149177"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "8f761ca31e7913e666fbca7d9c3f1c14"
server_key_iv: "b4151bd1c1cff8961f95c5405852b323"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.430227 29920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.438151 29920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:37.439924 29920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:37.443084 29920 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:37763
I20260808 07:11:37.443516 29920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
I20260808 07:11:37.443602 30110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:37763 every 8 connection(s)
I20260808 07:11:37.444494 30112 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:11:37.445045 30112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Bootstrap starting.
I20260808 07:11:37.448215 30112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:11:37.448480 30112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: Bootstrap complete.
I20260808 07:11:37.448691 30112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } }
I20260808 07:11:37.448796 30112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9f3f91336b2a4abf9dbdbc4652149177, State: Initialized, Role: LEARNER
I20260808 07:11:37.448836 30112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } }
I20260808 07:11:37.449008 30112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:37.449025 30115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } }
I20260808 07:11:37.449150 30115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:11:37.450608 30126 catalog_manager.cc:1259] Loaded cluster ID: ddfba8ff318b41969f8a285d353b4125
I20260808 07:11:37.450659 30126 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: loading cluster ID for follower catalog manager: success
I20260808 07:11:37.451781 30126 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: acquiring CA information for follower catalog manager: success
I20260808 07:11:37.452059 30126 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:11:37.461172 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.454680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.22.2.189:37763)
0808 07:11:37.454809 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.454811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.454817 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:11:37.454836 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.454837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.454845 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.454914 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.455461 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.456194 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.456910 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.457113 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.457188 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.458422 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.458423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.458423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.458430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.459991 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.460346 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.460348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.460348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.460409 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.460617 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.460618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.460618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.460744 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.460854 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.460962 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.461044 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
WARNING: no policy specified for oryx/127.22.2.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.188.keytab.
Entry for principal oryx/127.22.2.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.188.keytab.
I20260808 07:11:37.490262 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:45357
--webserver_interface=127.22.2.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.188.keytab
--principal=oryx/127.22.2.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41569,127.22.2.189:37763,127.22.2.188:45357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:37.574931 30132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:37.575203 30132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:37.575227 30132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:37.576368 30132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:37.576402 30132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:37.576417 30132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:37.576431 30132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:37.576442 30132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:37.577831 30132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41569,127.22.2.189:37763,127.22.2.188:45357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.188.keytab
--principal=oryx/127.22.2.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:45357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.2.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:37.578024 30132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:37.578204 30132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:37.583230 30138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.583222 30137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.583346 30132 server_base.cc:1034] running on GCE node
W20260808 07:11:37.583222 30140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.583760 30132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:37.584055 30132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:37.585218 30132 hybrid_clock.cc:648] HybridClock initialized: now 1786173097585197 us; error 36 us; skew 500 ppm
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:37.587779 30132 init.cc:377] Logged in from keytab as oryx/127.22.2.188@KRBTEST.COM (short username oryx)
I20260808 07:11:37.588191 30132 webserver.cc:460] Webserver started at http://127.22.2.188:36417/ using document root <none> and password file <none>
I20260808 07:11:37.588423 30132 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.588495 30132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.588647 30132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:37.589694 30132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data/instance:
uuid: "b2a30a39bb284a39b15f828288bfd57b"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "bb28434be597feb0461ac72f8e74b55c"
server_key_iv: "326aa871882a6883ab0752f6b4ee6a89"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.590054 30132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal/instance:
uuid: "b2a30a39bb284a39b15f828288bfd57b"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "bb28434be597feb0461ac72f8e74b55c"
server_key_iv: "326aa871882a6883ab0752f6b4ee6a89"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.591419 30132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:37.592185 30147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.592366 30132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.592465 30132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
uuid: "b2a30a39bb284a39b15f828288bfd57b"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "bb28434be597feb0461ac72f8e74b55c"
server_key_iv: "326aa871882a6883ab0752f6b4ee6a89"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.592552 30132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.611598 30132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:37.613976 30132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:37.614090 30132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:37.618227 30132 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:45357
I20260808 07:11:37.618275 30203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:45357 every 8 connection(s)
I20260808 07:11:37.618693 30132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
I20260808 07:11:37.619745 30204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:37.621343 30204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:37.625767 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 30132
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:37.630816 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.622121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44186 (local address 127.22.2.190:41569)
0808 07:11:37.622165 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.622167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.622285 (+ 118us) server_negotiation.cc:407] Connection header received
0808 07:11:37.622700 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.622702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.622714 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.622804 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.623391 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.623941 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.625031 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.625214 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.625265 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.627743 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.627744 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.627745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.627752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.629171 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.629625 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.629626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.629626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.629726 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.630086 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.630087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.630088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.630243 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.630334 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.630611 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.630714 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:37.630992 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.621976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:44186)
0808 07:11:37.622152 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.622191 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.622607 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.622874 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.622877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.623104 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.623291 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.623294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.624025 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.624026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.624951 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.624954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.625295 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.625296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.625404 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.625920 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.625935 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.627650 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.629273 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.629279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.629281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.629550 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.629798 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.629799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.629800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.629993 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.630330 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.630335 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.630439 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.630550 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.630719 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260808 07:11:37.631547 30204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 07:11:37.634923 30209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.626390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.22.2.188:45357)
0808 07:11:37.626555 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.626559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.626579 (+ 20us) server_negotiation.cc:407] Connection header received
0808 07:11:37.626596 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.626597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.626612 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.626675 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.627405 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.628247 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.628980 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.629271 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.629357 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.632168 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.632174 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.632175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.632181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.633649 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.634027 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.634029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.634029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.634090 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.634305 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.634306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.634306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.634487 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.634643 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.634719 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.634830 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 07:11:37.637957 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.631829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.0.0.1:48306)
0808 07:11:37.631939 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.631951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.632003 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.632124 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.632124 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.632260 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.632302 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.632304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.633237 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.633238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.633757 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.633759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.634171 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.634172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.634258 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.634644 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.634649 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.636102 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.637298 (+ 1196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.637298 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.637299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.637457 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.637585 (+ 128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.637586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.637586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.637624 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.637777 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.637777 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.637805 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.637849 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.637906 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:11:37.637979 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.631875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.22.2.189:37763)
0808 07:11:37.631921 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.631924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.631990 (+ 66us) server_negotiation.cc:407] Connection header received
0808 07:11:37.632066 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.632067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.632077 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.632119 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.632378 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.633155 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.633842 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.634092 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.634159 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.636176 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.636177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.636177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.636183 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.637231 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.637511 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.637512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.637512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.637550 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.637654 (+ 104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.637655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.637655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.637743 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.637786 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.637862 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.637920 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:37.638453 30204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 07:11:37.645190 30209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.638757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48228 (local address 127.22.2.188:45357)
0808 07:11:37.638786 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.638788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.638796 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:37.638864 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.638864 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.638874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.638927 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.639155 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.639665 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.640278 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.640485 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.640567 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.642118 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.642119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.642119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.642125 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.644062 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.644453 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.644454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.644454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.644510 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.644703 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.644704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.644705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.644810 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.644883 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.645018 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.645094 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:11:37.645275 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.638707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:45357 (local address 127.0.0.1:48228)
0808 07:11:37.638753 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.638764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.638808 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.638930 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.638931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.639056 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.639078 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.639079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.639737 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.639738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.640186 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.640187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.640567 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.640568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.640614 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.641016 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.641020 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.642041 (+ 1021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.644194 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.644195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.644196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.644367 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.644576 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.644577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.644577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.644630 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.644875 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.644875 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.644934 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.645014 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.645148 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":58}
I20260808 07:11:37.646675 30204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Bootstrap starting.
I20260808 07:11:37.647526 30204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:37.647871 30204 log.cc:824] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:37.648686 30204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: No bootstrap required, opened a new log
I20260808 07:11:37.650574 30204 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } }
I20260808 07:11:37.650692 30204 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:37.650713 30204 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a30a39bb284a39b15f828288bfd57b, State: Initialized, Role: FOLLOWER
I20260808 07:11:37.650852 30204 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } }
I20260808 07:11:37.651117 30211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } } }
I20260808 07:11:37.651161 30204 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:37.651237 30211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:11:37.652820 30224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:37.652879 30224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:37.655723 30132 master_runner.cc:428] Detected that this master b2a30a39bb284a39b15f828288bfd57b is joining an existing cluster
I20260808 07:11:37.655767 30132 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.2.188:45357
I20260808 07:11:37.663738 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.656876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.22.2.190:41569)
0808 07:11:37.656963 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.656965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.656976 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:37.657002 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.657003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.657014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.657069 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.657524 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.658323 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.659460 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.659781 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.659818 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.660848 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.660850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.660850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.660858 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.662445 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.662854 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.662855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.662856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.662922 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.663125 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.663126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.663127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.663260 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.663332 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.663526 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.663645 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 07:11:37.663774 30230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.656714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:44200)
0808 07:11:37.656895 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.656905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.656956 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.657124 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.657125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.657345 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.657430 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.657432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.658426 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.658427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.659367 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.659368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.659924 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.659933 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.659995 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.660421 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.660425 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.660749 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.662560 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.662561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.662561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.662766 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.662996 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.662997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.662997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.663056 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.663334 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.663335 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.663406 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.663478 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.663694 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":6,"thread_start_us":101,"threads_started":1}
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 07:11:37.684145 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.656753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48318 (local address 127.22.2.189:37763)
0808 07:11:37.656807 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.656810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.657241 (+ 431us) server_negotiation.cc:407] Connection header received
0808 07:11:37.657339 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.657340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.657350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.657380 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.657833 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.658583 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.659694 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.660001 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.660045 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.680899 (+ 20854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.680901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.680901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.680911 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.682762 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683279 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.683281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.683281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.683335 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683575 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.683576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.683576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.683731 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.683817 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.683962 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.684045 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:11:37.684159 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.656717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.0.0.1:48318)
0808 07:11:37.657176 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.657185 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.657267 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.657445 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.657446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.657659 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.657727 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.657729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.658682 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.658683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.659595 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.659598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.660127 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.660127 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.660195 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.680146 (+ 19951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.680150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.680783 (+ 633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.682869 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.682870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.682870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.683197 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.683406 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.683407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.683483 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.683806 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.683807 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.683851 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.683936 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.684048 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":91,"threads_started":1}
I20260808 07:11:37.684299 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.665926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:45357 (local address 127.0.0.1:48252)
0808 07:11:37.665980 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.665993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.666061 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.666442 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.666443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.666654 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.666695 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.666698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.667651 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.667652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.668138 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.668139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.668610 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.668610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.668656 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.669044 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.669048 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.670353 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.682982 (+ 12629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.682984 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.682984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.683199 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.683445 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.683447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.683506 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.684001 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.684002 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.684046 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.684094 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.684199 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:11:37.684630 30209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.657083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.22.2.188:45357)
0808 07:11:37.657117 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.657118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.657293 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:11:37.657374 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.657375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.657383 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.657421 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.657834 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.658497 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.659452 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.659755 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.659805 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.680900 (+ 21095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.680901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.680902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.680908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.682777 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683276 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.683277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.683278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.683337 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683616 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.683617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.683617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.683752 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.683825 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.684428 (+ 603us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.684513 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:37.684777 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.666061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.22.2.188:45357)
0808 07:11:37.666263 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.666264 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.666273 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:37.666317 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.666317 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.666327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.666396 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.666812 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.667582 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.668215 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.668538 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.668606 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.670456 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.670457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.670458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.670465 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.682879 (+ 12414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683278 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.683279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.683280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.683367 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683598 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.683599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.683599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.683924 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.684466 (+ 542us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.684506 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.684583 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":138,"threads_started":1}
I20260808 07:11:37.684899 30231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.657078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:45357 (local address 127.0.0.1:48242)
0808 07:11:37.657235 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.657245 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.657312 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.657469 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.657469 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.657695 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.657754 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.657756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.658568 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.658568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.659366 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.659367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.659945 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.659945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.659998 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.680262 (+ 20264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.680268 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.680818 (+ 550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.682854 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.682855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.682855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.683197 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.683405 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.683405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.683405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.683497 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.684251 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.684252 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.684297 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.684720 (+ 423us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.684808 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":3,"thread_start_us":68,"threads_started":1}
I20260808 07:11:37.685672 29397 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.2.188:45357: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b"
member_type: NON_VOTER
last_known_addr {
host: "127.22.2.188"
port: 45357
}
attrs {
promote: true
}
}
dest_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5"
cas_config_opid_index: 5
I20260808 07:11:37.685945 29397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }
I20260808 07:11:37.686304 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 29468
I20260808 07:11:37.686429 30027 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b2a30a39bb284a39b15f828288bfd57b (127.22.2.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.687417 30027 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.2.188:45357
I20260808 07:11:37.687500 30027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.687615 30027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:37.690104 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.686391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.0.0.1:48330)
0808 07:11:37.687466 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.687479 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.687530 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.687825 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.687826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.687828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:37.687867 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.687869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.688554 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.688555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.689303 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.689305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.689860 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.689861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.689918 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.689942 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.690002 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.690039 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060}
I20260808 07:11:37.690102 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.686840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.22.2.189:37763)
0808 07:11:37.687435 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.687437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.687617 (+ 180us) server_negotiation.cc:407] Connection header received
0808 07:11:37.687638 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.687639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.687683 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.687763 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:37.687940 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.688465 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.689393 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.689777 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.689861 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.689933 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.690008 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.690039 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573}
I20260808 07:11:37.690768 30079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 LEARNER]: Refusing update from remote peer 9ed2c8af33b24d7a9ae45e99e07787c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:11:37.690985 30025 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 07:11:37.691797 30079 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b2a30a39bb284a39b15f828288bfd57b (127.22.2.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.693130 30236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.693220 30236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:11:37.693598 30025 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: attempting to promote NON_VOTER 9f3f91336b2a4abf9dbdbc4652149177 to VOTER
I20260808 07:11:37.693787 30025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }
I20260808 07:11:37.694432 30079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 LEARNER]: Refusing update from remote peer 9ed2c8af33b24d7a9ae45e99e07787c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 07:11:37.694569 30027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 07:11:37.695118 30027 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.696036 30027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.696110 30027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:37.696152 30027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.696213 30027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:37.696527 30079 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.697489 30236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.697570 30236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:37.698028 30115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ed2c8af33b24d7a9ae45e99e07787c5. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.698062 30177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:37.698110 30115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:37.698894 30177 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Refusing update from remote peer 9ed2c8af33b24d7a9ae45e99e07787c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:11:37.699116 30176 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Refusing update from remote peer 9ed2c8af33b24d7a9ae45e99e07787c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 07:11:37.699294 30027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:37.700762 30176 raft_consensus.cc:858] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }
W20260808 07:11:37.700825 30176 raft_consensus.cc:1565] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }
I20260808 07:11:37.702051 30176 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } }
I20260808 07:11:37.703114 30211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ed2c8af33b24d7a9ae45e99e07787c5. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } }
I20260808 07:11:37.703214 30211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 07:11:37.703285 30211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } }
I20260808 07:11:37.703334 30211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 07:11:37.703540 30027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.703611 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } }
I20260808 07:11:37.703651 30027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:37.703682 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 07:11:37.703882 30176 raft_consensus.cc:1467] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260808 07:11:37.704017 30176 raft_consensus.cc:858] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 LEARNER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } }.
Pending config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }:
New config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }
W20260808 07:11:37.704078 30176 raft_consensus.cc:1565] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 LEARNER]: Could not prepare op 'term: 1 index: 7' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } }.
Pending config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }:
New config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }
I20260808 07:11:37.704228 30176 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b2a30a39bb284a39b15f828288bfd57b (127.22.2.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.704917 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.704991 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: LEARNER
I20260808 07:11:37.705031 30025 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: attempting to promote NON_VOTER b2a30a39bb284a39b15f828288bfd57b to VOTER
I20260808 07:11:37.705127 30025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:37.705504 30079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Refusing update from remote peer 9ed2c8af33b24d7a9ae45e99e07787c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 07:11:37.705524 30177 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 LEARNER]: Refusing update from remote peer 9ed2c8af33b24d7a9ae45e99e07787c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 07:11:37.705684 30027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 07:11:37.705757 30025 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 07:11:37.706059 30177 raft_consensus.cc:858] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 LEARNER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }.
Pending config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }:
New config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
W20260808 07:11:37.706103 30177 raft_consensus.cc:1565] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 LEARNER]: Could not prepare op 'term: 1 index: 8' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: NON_VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: true } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }.
Pending config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } }:
New config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:37.706173 30177 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 LEARNER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: NON_VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: true } } }
I20260808 07:11:37.706269 30027 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b2a30a39bb284a39b15f828288bfd57b (127.22.2.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.707139 30027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.707239 30027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:37.707333 30027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.707387 30027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:37.707433 30027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.707494 30027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:37.707587 30177 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b2a30a39bb284a39b15f828288bfd57b (127.22.2.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.707746 30079 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b2a30a39bb284a39b15f828288bfd57b (127.22.2.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.708333 30211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.708436 30211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:37.708477 30211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.708496 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:46071
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:41569
--webserver_port=46369
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763,127.22.2.188:45357 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:37.708531 30211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:37.708819 30115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.708916 30115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:37.709249 30176 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 07:11:37.710065 30132 master.cc:537] Master@127.22.2.188:45357 shutting down...
I20260808 07:11:37.713001 30132 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:37.713076 30132 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:37.713111 30132 tablet_replica.cc:330] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: stopping tablet replica
I20260808 07:11:37.726708 30132 master.cc:552] Master@127.22.2.188:45357 shutdown complete.
I20260808 07:11:37.726779 30132 master_runner.cc:305] Clearing existing system tablet
I20260808 07:11:37.726929 30132 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.726977 30132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.727782 30132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:37.728387 30249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.728555 30132 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.728619 30132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.728643 30132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
uuid: "b2a30a39bb284a39b15f828288bfd57b"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "bb28434be597feb0461ac72f8e74b55c"
server_key_iv: "326aa871882a6883ab0752f6b4ee6a89"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.729410 30132 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:37.730943 30132 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:11:37.731016 30132 log.cc:1196] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 07:11:37.731254 30132 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Deleting consensus metadata
I20260808 07:11:37.731386 30132 master_runner.cc:315] Copying system tablet from 127.22.2.190:41569
I20260808 07:11:37.732070 30132 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: tablet copy: Beginning tablet copy session from remote peer at address 127.22.2.190:41569
I20260808 07:11:37.739305 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.732450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.22.2.190:41569)
0808 07:11:37.732497 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.732499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.732559 (+ 60us) server_negotiation.cc:407] Connection header received
0808 07:11:37.732628 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.732629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.732640 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.732727 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.733066 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.733841 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.734885 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.735178 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.735250 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.736423 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.736425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.736425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.736432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.738066 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.738488 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.738489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.738490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.738555 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.738774 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.738775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.738775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.738906 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.738973 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.739135 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.739216 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:11:37.739641 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.732369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:44202)
0808 07:11:37.732500 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.732512 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.732572 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.732734 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.732735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.732937 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.732991 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.732993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.733931 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.733932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.734795 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.734797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.735267 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.735269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.735336 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.735883 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.735917 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.736326 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.738158 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.738159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.738159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.738404 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.738630 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.738630 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.738631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.738698 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.738976 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.738977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.739025 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.739095 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.739196 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260808 07:11:37.739878 29420 tablet_copy_service.cc:140] P 9ed2c8af33b24d7a9ae45e99e07787c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b2a30a39bb284a39b15f828288bfd57b ({username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:44202)
I20260808 07:11:37.739938 29420 tablet_copy_service.cc:161] P 9ed2c8af33b24d7a9ae45e99e07787c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b2a30a39bb284a39b15f828288bfd57b at {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:44202: session id = b2a30a39bb284a39b15f828288bfd57b-00000000000000000000000000000000
I20260808 07:11:37.740514 29420 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:11:37.741008 30132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:11:37.742154 30132 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: tablet copy: Starting download of 0 data blocks...
I20260808 07:11:37.742300 30132 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: tablet copy: Starting download of 1 WAL segments...
I20260808 07:11:37.743067 30132 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:11:37.744006 29420 tablet_copy_service.cc:342] P 9ed2c8af33b24d7a9ae45e99e07787c5: Request end of tablet copy session b2a30a39bb284a39b15f828288bfd57b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:44202
I20260808 07:11:37.744061 29420 tablet_copy_service.cc:433] P 9ed2c8af33b24d7a9ae45e99e07787c5: ending tablet copy session b2a30a39bb284a39b15f828288bfd57b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b2a30a39bb284a39b15f828288bfd57b
I20260808 07:11:37.746483 30132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:37.747759 30262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.747756 30259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.748019 30132 server_base.cc:1034] running on GCE node
W20260808 07:11:37.751708 30260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.751888 30132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:37.752038 30132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:37.753170 30132 hybrid_clock.cc:648] HybridClock initialized: now 1786173097753143 us; error 43 us; skew 500 ppm
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:37.754685 30132 init.cc:377] Logged in from keytab as oryx/127.22.2.188@KRBTEST.COM (short username oryx)
I20260808 07:11:37.754951 30132 webserver.cc:460] Webserver started at http://127.22.2.188:37447/ using document root <none> and password file <none>
I20260808 07:11:37.755074 30132 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.755110 30132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.755995 30132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:11:37.756654 30269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.756860 30132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:37.756918 30132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
uuid: "b2a30a39bb284a39b15f828288bfd57b"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "bb28434be597feb0461ac72f8e74b55c"
server_key_iv: "326aa871882a6883ab0752f6b4ee6a89"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.756983 30132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.785362 30132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:37.787374 30132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:37.790519 30132 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:45357
I20260808 07:11:37.790618 30325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:45357 every 8 connection(s)
I20260808 07:11:37.790872 30132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
I20260808 07:11:37.791911 30326 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:11:37.792480 30326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Bootstrap starting.
I20260808 07:11:37.795163 30326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:11:37.795397 30326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Bootstrap complete.
I20260808 07:11:37.795637 30326 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:37.795737 30326 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a30a39bb284a39b15f828288bfd57b, State: Initialized, Role: FOLLOWER
I20260808 07:11:37.795835 30326 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:37.796013 30326 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:37.797679 30341 catalog_manager.cc:1259] Loaded cluster ID: ddfba8ff318b41969f8a285d353b4125
I20260808 07:11:37.797796 30341 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: loading cluster ID for follower catalog manager: success
I20260808 07:11:37.798786 30330 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:37.798884 30330 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:37.799266 30341 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: acquiring CA information for follower catalog manager: success
I20260808 07:11:37.799639 30341 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 07:11:37.808660 30247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:37.808861 30247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:37.808887 30247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:37.810160 30247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:37.810204 30247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:37.810232 30247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:37.812090 30247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:46071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=46369
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763,127.22.2.188:45357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:37.812383 30247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:37.812649 30247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:37.818845 30346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.818907 30347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.818871 30349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.819255 30247 server_base.cc:1034] running on GCE node
I20260808 07:11:37.819504 30247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:37.819793 30247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:37.820986 30247 hybrid_clock.cc:648] HybridClock initialized: now 1786173097820960 us; error 40 us; skew 500 ppm
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:37.823827 30247 init.cc:377] Logged in from keytab as oryx/127.22.2.129@KRBTEST.COM (short username oryx)
I20260808 07:11:37.824365 30247 webserver.cc:460] Webserver started at http://127.22.2.129:46369/ using document root <none> and password file <none>
I20260808 07:11:37.824621 30247 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.824712 30247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.826234 30247 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:37.826929 30358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.827112 30247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:37.827226 30247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "97880488f60b4581b7f9fb132157599b"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "c0f2f2e7edbe0de2b77123846118c58b"
server_key_iv: "be231a65bbe32ef8651f37224c62d5b6"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.827651 30247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.842376 30247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:37.844931 30247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:37.845119 30247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:37.845363 30247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:37.845717 30247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:37.845784 30247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.845835 30247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:37.845860 30247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.853003 30247 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:46071
I20260808 07:11:37.853040 30483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:46071 every 8 connection(s)
I20260808 07:11:37.853417 30247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:37.854709 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 30247
I20260808 07:11:37.854890 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 29618
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
I20260808 07:11:37.864948 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:38215
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:41569
--webserver_port=34583
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763,127.22.2.188:45357 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:37.868522 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.854274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34895 (local address 127.22.2.190:41569)
0808 07:11:37.854332 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.854335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.854542 (+ 207us) server_negotiation.cc:407] Connection header received
0808 07:11:37.856959 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.856961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.856978 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.857027 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.858065 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.858880 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.860560 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.860878 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.860928 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.864712 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.864714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.864714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.864722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.866731 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867221 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.867222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.867223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.867284 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867608 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.867609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.867610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.867751 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.867811 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.868325 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.868419 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:11:37.868527 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.854511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:39347 (local address 127.22.2.188:45357)
0808 07:11:37.854671 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.854673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.854683 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:37.856989 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.856990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.857006 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.857085 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.858063 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.859101 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.860555 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.860880 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.860942 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.864237 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.864239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.864239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.864247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.867008 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867474 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.867475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.867476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.867543 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867854 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.867855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.867856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.868006 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.868072 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.868342 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.868430 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":61,"threads_started":1}
I20260808 07:11:37.868583 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.854155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:34895)
0808 07:11:37.854459 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.854469 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.856806 (+ 2337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.857192 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.857197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.857683 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.857941 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.857948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.859029 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.859030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.860442 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.860445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.861019 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.861020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.861164 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.861942 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.861947 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.864612 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.866801 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.866808 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.866811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.867124 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.867373 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.867374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.867505 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.867846 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.867853 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.868008 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.868133 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.868337 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":104,"thread_start_us":95,"threads_started":1}
I20260808 07:11:37.868839 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.854436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:59347 (local address 127.22.2.189:37763)
0808 07:11:37.854641 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:37.854643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:37.854655 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:11:37.859422 (+ 4767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:37.859424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:37.859440 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:37.859478 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:37.859951 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.860726 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.861901 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.862240 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.862303 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.865102 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:37.865103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:37.865104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:37.865112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:37.867487 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867931 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.867932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.867932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.867993 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:37.868218 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:37.868219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:37.868219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:37.868368 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:37.868423 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:37.868604 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 07:11:37.868691 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260808 07:11:37.869125 30485 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41569
I20260808 07:11:37.869234 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.854155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.22.2.129:59347)
0808 07:11:37.854454 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.854466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.859293 (+ 4827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.859577 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.859578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.859771 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.859855 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.859857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.860840 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.860841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.861784 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.861788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.862385 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.862386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.862455 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.863191 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.863196 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.864948 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.867588 (+ 2640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.867589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.867828 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.868069 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.868070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.868070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.868136 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.868456 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.868457 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.868495 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.868914 (+ 419us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.869022 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":3,"thread_start_us":70,"threads_started":1}
I20260808 07:11:37.869256 30485 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:37.869452 30485 heartbeater.cc:511] Master 127.22.2.190:41569 requested a full tablet report, sending...
I20260808 07:11:37.869871 30487 heartbeater.cc:347] Connected to a master server at 127.22.2.189:37763
I20260808 07:11:37.869868 29397 ts_manager.cc:194] Re-registered known tserver with Master: 97880488f60b4581b7f9fb132157599b (127.22.2.129:46071)
I20260808 07:11:37.869939 30487 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:37.870020 30487 heartbeater.cc:511] Master 127.22.2.189:37763 requested a full tablet report, sending...
I20260808 07:11:37.870352 29397 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:34895
I20260808 07:11:37.870431 30069 ts_manager.cc:194] Registered new tserver with Master: 97880488f60b4581b7f9fb132157599b (127.22.2.129:46071)
I20260808 07:11:37.871923 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:37.854155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:45357 (local address 127.22.2.129:39347)
0808 07:11:37.854412 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:37.854454 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:37.856805 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:37.857192 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:37.857197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:37.857653 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:37.857941 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.857948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.859200 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.859201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.860442 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:37.860445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:37.861012 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:37.861013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:37.861164 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:37.861843 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:37.861854 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:37.864116 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:37.867120 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.867122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.867379 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.867681 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:37.867682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:37.867682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:37.867775 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:37.868102 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:37.868103 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:37.868149 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:11:37.868233 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:11:37.871763 (+ 3530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":3437,"thread_start_us":88,"threads_started":1}
I20260808 07:11:37.872474 30484 heartbeater.cc:347] Connected to a master server at 127.22.2.188:45357
I20260808 07:11:37.872536 30484 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:37.872614 30484 heartbeater.cc:511] Master 127.22.2.188:45357 requested a full tablet report, sending...
I20260808 07:11:37.872990 30287 ts_manager.cc:194] Registered new tserver with Master: 97880488f60b4581b7f9fb132157599b (127.22.2.129:46071)
W20260808 07:11:37.953229 30494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:37.953436 30494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:37.953460 30494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:37.954552 30494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:37.954587 30494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:37.954612 30494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:37.956192 30494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:38215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=34583
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763,127.22.2.188:45357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:37.956429 30494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:37.956619 30494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:37.961655 30503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.961656 30500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:37.961680 30501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:37.962049 30494 server_base.cc:1034] running on GCE node
I20260808 07:11:37.962246 30494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:37.962491 30494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:37.963615 30494 hybrid_clock.cc:648] HybridClock initialized: now 1786173097963607 us; error 35 us; skew 500 ppm
Aug 08 07:11:37 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:37.966207 30494 init.cc:377] Logged in from keytab as oryx/127.22.2.130@KRBTEST.COM (short username oryx)
I20260808 07:11:37.966595 30494 webserver.cc:460] Webserver started at http://127.22.2.130:34583/ using document root <none> and password file <none>
I20260808 07:11:37.966813 30494 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:37.966869 30494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:37.968267 30494 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:37.968822 30512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.969013 30494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:37.969081 30494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "0296fb38106f45de9a3b43df9b814368"
format_stamp: "Formatted at 2026-08-08 07:11:36 on dist-test-slave-6k22"
server_key: "9e9e48b6065f1964b9cef851493c4b8f"
server_key_iv: "3ff00ab2e78b516e6f936005b4bcb1db"
server_key_version: "encryptionkey@0"
I20260808 07:11:37.969388 30494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:37.993985 30494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:37.996496 30494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:37.996627 30494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:37.996853 30494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:37.997198 30494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:37.997252 30494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:37.997277 30494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:37.997289 30494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:38.003363 30494 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:38215
I20260808 07:11:38.003513 30642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:38215 every 8 connection(s)
I20260808 07:11:38.003780 30494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:11:38 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
Aug 08 07:11:38 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
Aug 08 07:11:38 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173097, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:38.012035 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 30494
I20260808 07:11:38.012238 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 29766
I20260808 07:11:38.014670 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.004752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:59357 (local address 127.22.2.188:45357)
0808 07:11:38.004830 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.004832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.004979 (+ 147us) server_negotiation.cc:407] Connection header received
0808 07:11:38.005511 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.005513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.005529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.005591 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.006322 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.007101 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008331 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.008542 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008627 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.010961 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.010962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.010963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.010970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.013005 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.013475 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.013476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.013477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.013548 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.013831 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.013832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.013833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.014007 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.014093 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.014396 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.014516 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:11:38.014972 30647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.004505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:45357 (local address 127.22.2.130:59357)
0808 07:11:38.004866 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.004901 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.005409 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.005662 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.005666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.006026 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:38.006220 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.006225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.007629 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.007630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.008217 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.008219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.008639 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.008760 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.009278 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:38.009289 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:38.010878 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:38.013105 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.013112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.013114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.013399 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.013622 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.013622 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.013623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.013750 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.014085 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:38.014090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:38.014244 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.014347 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.014501 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":87,"threads_started":1}
I20260808 07:11:38.016045 30643 heartbeater.cc:347] Connected to a master server at 127.22.2.188:45357
I20260808 07:11:38.016196 30643 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:38.016386 30643 heartbeater.cc:511] Master 127.22.2.188:45357 requested a full tablet report, sending...
I20260808 07:11:38.016834 30287 ts_manager.cc:194] Registered new tserver with Master: 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215)
I20260808 07:11:38.023175 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.004584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:40677 (local address 127.22.2.189:37763)
0808 07:11:38.004632 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.004635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.005155 (+ 520us) server_negotiation.cc:407] Connection header received
0808 07:11:38.005529 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.005530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.005544 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.005578 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.006341 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.006846 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008328 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.008631 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008673 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.011472 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.011473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.011474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.011482 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.021816 (+ 10334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.022220 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.022221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.022221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.022288 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.022558 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.022560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.022560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.022721 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.022769 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.022985 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.023079 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:38.023212 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.004506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.22.2.130:40677)
0808 07:11:38.005089 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.005096 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.005428 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.005683 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.005684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.006086 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:38.006221 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.006229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.006948 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.006949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.008219 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.008222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.008759 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.008855 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.009522 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:38.009526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:38.011348 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:38.021912 (+ 10564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.021913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.021913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.022134 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.022369 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.022370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.022370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.022478 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.022820 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:38.022821 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:38.022867 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.022937 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.023055 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":219,"thread_start_us":55,"threads_started":1}
I20260808 07:11:38.023599 30645 heartbeater.cc:347] Connected to a master server at 127.22.2.189:37763
I20260808 07:11:38.023653 30645 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:38.023732 30645 heartbeater.cc:511] Master 127.22.2.189:37763 requested a full tablet report, sending...
I20260808 07:11:38.024013 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.004505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:52407)
0808 07:11:38.004923 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.004933 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.005430 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.006041 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.006042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.006206 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:38.006262 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.006264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.007199 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.007200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.008219 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.008222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.008759 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.008855 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.009598 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:38.009603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:38.011981 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:38.022356 (+ 10375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.022357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.022357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.022531 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.022739 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.022740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.022740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.022804 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.023126 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:38.023127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:38.023167 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.023770 (+ 603us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.023898 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":15,"thread_start_us":79,"threads_started":1}
I20260808 07:11:38.024030 30069 ts_manager.cc:194] Registered new tserver with Master: 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215)
I20260808 07:11:38.024096 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.004646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:52407 (local address 127.22.2.190:41569)
0808 07:11:38.004699 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.004701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.005027 (+ 326us) server_negotiation.cc:407] Connection header received
0808 07:11:38.005530 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.005532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.005545 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.005578 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.006366 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.007107 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008329 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.008632 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.008673 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.012062 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.012063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.012063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.012071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.022279 (+ 10208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.022606 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.022608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.022608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.022671 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.022877 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.022877 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.022878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.023031 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.023097 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.023861 (+ 764us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.023956 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:11:38.024586 30644 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41569
I20260808 07:11:38.024641 30644 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:38.024708 30644 heartbeater.cc:511] Master 127.22.2.190:41569 requested a full tablet report, sending...
I20260808 07:11:38.024931 29397 ts_manager.cc:194] Re-registered known tserver with Master: 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215)
I20260808 07:11:38.025149 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:41063
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:41569
--webserver_port=34033
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763,127.22.2.188:45357 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:38.025429 29397 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:52407
W20260808 07:11:38.109090 30650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:38.109300 30650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:38.109325 30650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:38.110617 30650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:38.110659 30650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:38.110718 30650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:38.112358 30650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:41063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=34033
--tserver_master_addrs=127.22.2.190:41569,127.22.2.189:37763,127.22.2.188:45357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:38.112617 30650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:38.112910 30650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:38.118749 30659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:38.118794 30656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:38.118898 30650 server_base.cc:1034] running on GCE node
W20260808 07:11:38.118796 30655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:38.119155 30650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:38.119357 30650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:38.120492 30650 hybrid_clock.cc:648] HybridClock initialized: now 1786173098120460 us; error 46 us; skew 500 ppm
Aug 08 07:11:38 dist-test-slave-6k22 krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173098, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:38.123513 30650 init.cc:377] Logged in from keytab as oryx/127.22.2.131@KRBTEST.COM (short username oryx)
I20260808 07:11:38.124014 30650 webserver.cc:460] Webserver started at http://127.22.2.131:34033/ using document root <none> and password file <none>
I20260808 07:11:38.124254 30650 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:38.124342 30650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:38.125749 30650 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:11:38.126338 30669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:38.126533 30650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:38.126623 30650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "d26a25f00305411abd569f0a54ea6cbf"
format_stamp: "Formatted at 2026-08-08 07:11:37 on dist-test-slave-6k22"
server_key: "5d232936f53c93e6844ca132d54e3f0e"
server_key_iv: "db6c2e12e5671bf6aef4ad25ace1090f"
server_key_version: "encryptionkey@0"
I20260808 07:11:38.126953 30650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:38.138046 30650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:38.139818 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.136724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:45357 (local address 127.0.0.1:48254)
0808 07:11:38.136818 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.136829 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.136873 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.137052 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.137053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.137054 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:38.137096 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.137098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.137966 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.137967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.138732 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.138734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.139552 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.139588 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.139653 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.139656 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.139723 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.139763 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 07:11:38.139854 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.136802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.22.2.188:45357)
0808 07:11:38.136865 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.136867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.136878 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:38.136954 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.136955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.136979 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.137040 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:38.137153 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.137893 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.138818 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.139457 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.139531 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.139616 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.139734 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.139774 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:11:38.140213 30330 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ed2c8af33b24d7a9ae45e99e07787c5. Latest consensus state: current_term: 1 leader_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:38.140359 30330 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:38.140583 30650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:38.140743 30650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:38.140952 30650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:38.141254 30650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:38.141321 30650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:38.141362 30650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:38.141390 30650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:38.147276 30650 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:41063
I20260808 07:11:38.147359 30794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:41063 every 8 connection(s)
I20260808 07:11:38.147746 30650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:38.150269 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 30650
Aug 08 07:11:38 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173098, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
Aug 08 07:11:38 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173098, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.189@KRBTEST.COM
Aug 08 07:11:38 dist-test-slave-6k22 krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173098, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.188@KRBTEST.COM
I20260808 07:11:38.160382 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.148684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:56127 (local address 127.22.2.188:45357)
0808 07:11:38.148749 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.148751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.148760 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:38.149974 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.149975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.149991 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.150039 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.151745 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.152441 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.153517 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.154309 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.154369 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.157132 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.157133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.157133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.157141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.158776 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.159387 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.159388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.159388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.159445 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.159663 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.159664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.159664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.159785 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.159840 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.160198 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.160276 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:11:38.160863 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.148939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:55153 (local address 127.22.2.189:37763)
0808 07:11:38.148991 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.149010 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.149032 (+ 22us) server_negotiation.cc:407] Connection header received
0808 07:11:38.149996 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.149997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.150011 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.150060 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.150933 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.151650 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.153063 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.153369 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.153422 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.156758 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.156760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.156760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.156768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.158622 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.159155 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.159156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.159157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.159228 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.159535 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.159536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.159536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.159722 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.159786 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.160647 (+ 861us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.160738 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:11:38.161140 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.148869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.22.2.131:55153)
0808 07:11:38.148979 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.148989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.149877 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.150153 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.150153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.150562 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:38.150797 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.150812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.151748 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.151749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.152974 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.152976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.153478 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.153478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.153601 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.154114 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:38.154124 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:38.156637 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:38.158733 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.158741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.158743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.159072 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.159309 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.159310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.159311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.159454 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.159823 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:38.159830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:38.159994 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.160109 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.160293 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":3,"thread_start_us":66,"threads_started":1}
I20260808 07:11:38.161145 30800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.148431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:45357 (local address 127.22.2.131:56127)
0808 07:11:38.148623 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.148666 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.149876 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.150140 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.150145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.150550 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:38.150796 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.150802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.152528 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.152529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.153429 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.153431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.154421 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.154422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.154486 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.154872 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:38.154877 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:38.157046 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:38.159103 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.159104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.159105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.159297 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.159512 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.159512 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.159513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.159597 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.159863 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:38.159864 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:38.159992 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.160128 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.160589 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":344,"spinlock_wait_cycles":7936,"thread_start_us":125,"threads_started":1}
I20260808 07:11:38.162729 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.148400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:47005 (local address 127.22.2.190:41569)
0808 07:11:38.148450 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.148453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.148744 (+ 291us) server_negotiation.cc:407] Connection header received
0808 07:11:38.149994 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.149995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.150008 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.150042 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.151630 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.152362 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.153348 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.153613 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.153652 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.158858 (+ 5206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.158859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.158859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.158866 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.160608 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.161815 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.161816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.161817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.161882 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.162106 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.162107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.162108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.162261 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.162309 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.162531 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.162613 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:38.163069 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.148431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:47005)
0808 07:11:38.148624 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.148666 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.149877 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.151261 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.151262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.151434 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:38.151506 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.151508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.152461 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.152463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.153248 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.153250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.153730 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.153731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.153800 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.154261 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:38.154265 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:38.157949 (+ 3684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:38.160939 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.160940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.160940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.161686 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.161960 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:38.161961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:38.161961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:38.162025 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:38.162361 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:38.162362 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:38.162407 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.162835 (+ 428us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.162941 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260808 07:11:38.163285 30796 heartbeater.cc:347] Connected to a master server at 127.22.2.188:45357
I20260808 07:11:38.163381 30796 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:38.163451 30797 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41569
I20260808 07:11:38.163493 30797 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:38.163631 30797 heartbeater.cc:511] Master 127.22.2.190:41569 requested a full tablet report, sending...
I20260808 07:11:38.163661 30798 heartbeater.cc:347] Connected to a master server at 127.22.2.189:37763
I20260808 07:11:38.163699 30798 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:38.163748 30798 heartbeater.cc:511] Master 127.22.2.189:37763 requested a full tablet report, sending...
I20260808 07:11:38.164021 29397 ts_manager.cc:194] Re-registered known tserver with Master: d26a25f00305411abd569f0a54ea6cbf (127.22.2.131:41063)
I20260808 07:11:38.164119 30069 ts_manager.cc:194] Registered new tserver with Master: d26a25f00305411abd569f0a54ea6cbf (127.22.2.131:41063)
I20260808 07:11:38.164515 29397 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:47005
I20260808 07:11:38.164644 30796 heartbeater.cc:511] Master 127.22.2.188:45357 requested a full tablet report, sending...
I20260808 07:11:38.164950 30287 ts_manager.cc:194] Registered new tserver with Master: d26a25f00305411abd569f0a54ea6cbf (127.22.2.131:41063)
I20260808 07:11:38.165434 30807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.151316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.22.2.190:41569)
0808 07:11:38.151600 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.151602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.153817 (+ 2215us) server_negotiation.cc:407] Connection header received
0808 07:11:38.153933 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.153934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.153970 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.155658 (+ 1688us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.155702 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.157727 (+ 2025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.160122 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.160411 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.160705 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.162110 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.162112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.162112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.162119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.164101 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.164505 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.164506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.164506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.164568 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.164769 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.164770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.164770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.164897 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.164992 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.165214 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.165306 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":61,"threads_started":1}
I20260808 07:11:38.166118 30813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.151827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.22.2.188:45357)
0808 07:11:38.152053 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.152055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.153208 (+ 1153us) server_negotiation.cc:407] Connection header received
0808 07:11:38.153229 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.153230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.153241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.153289 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.153969 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.154667 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.156760 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.157334 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.158134 (+ 800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.161301 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.161302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.161302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.161309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.164446 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.165251 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.165252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.165252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.165313 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.165584 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.165585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.165585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.165706 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.165809 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.165918 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.165998 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":52,"threads_started":1}
I20260808 07:11:38.167140 30810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.151473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.22.2.189:37763)
0808 07:11:38.152231 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.152233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.152240 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:38.152258 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.152259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.152269 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.152318 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.154779 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.155538 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.156758 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.157098 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.157160 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.159130 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.159131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.159132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.159139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.163104 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.166052 (+ 2948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.166053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.166053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.166117 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.166575 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.166576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.166577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.166740 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.166859 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.166959 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.167050 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":46,"threads_started":1}
I20260808 07:11:38.167153 29397 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:38.168867 29397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:38.176105 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.172141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46071 (local address 127.0.0.1:53338)
0808 07:11:38.172174 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.172187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.172227 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.172672 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.172673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.172674 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:38.172728 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.172729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.173926 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.173928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.174949 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.174952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.175914 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.175916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.175975 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.175979 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.176006 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.176044 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 07:11:38.176548 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.172297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53338 (local address 127.22.2.129:46071)
0808 07:11:38.172488 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.172493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.172504 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:38.172548 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.172550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.172567 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.172810 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:38.172868 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.173821 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.175052 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.175819 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.176297 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.176409 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.176434 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.176472 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
I20260808 07:11:38.179669 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.172185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:38215 (local address 127.0.0.1:36146)
0808 07:11:38.172702 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.172713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.172767 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.173771 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.173772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.173774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:38.173894 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.173896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.175419 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.175420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.178171 (+ 2751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.178175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.179419 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.179420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.179484 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.179488 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.179537 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.179603 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":88,"threads_started":1}
I20260808 07:11:38.179908 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.172383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:41063 (local address 127.0.0.1:51622)
0808 07:11:38.176171 (+ 3788us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.176180 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.176219 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.176468 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.176469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.176470 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:38.176523 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.176524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.177653 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.177654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.178762 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.178766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.179715 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.179716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.179778 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.179781 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.179808 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.179841 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3779,"thread_start_us":68,"threads_started":1}
I20260808 07:11:38.180018 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.172278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.22.2.130:38215)
0808 07:11:38.173462 (+ 1184us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.173467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.173518 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:11:38.173582 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.173585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.173602 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.173733 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:38.174016 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.175319 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.178262 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.178984 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.179052 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.179382 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.179759 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.179936 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":98,"threads_started":1}
I20260808 07:11:38.180202 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.172556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.22.2.131:41063)
0808 07:11:38.173392 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.173503 (+ 111us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.176251 (+ 2748us) server_negotiation.cc:407] Connection header received
0808 07:11:38.176299 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.176301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.176342 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.176613 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:38.176671 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.177583 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.178857 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.179610 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.179988 (+ 378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.180064 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.180088 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.180127 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":74,"threads_started":1}
I20260808 07:11:38.182835 30413 tablet_service.cc:1467] Processing CreateTablet for tablet f4a7f11ce41d44a6b12fb6040d580258 (DEFAULT_TABLE table=test-table [id=ac355b0bad624d63bd177b74735cf682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:38.182833 30725 tablet_service.cc:1467] Processing CreateTablet for tablet f4a7f11ce41d44a6b12fb6040d580258 (DEFAULT_TABLE table=test-table [id=ac355b0bad624d63bd177b74735cf682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:38.182843 30569 tablet_service.cc:1467] Processing CreateTablet for tablet f4a7f11ce41d44a6b12fb6040d580258 (DEFAULT_TABLE table=test-table [id=ac355b0bad624d63bd177b74735cf682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:38.183214 30569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a7f11ce41d44a6b12fb6040d580258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:38.183213 30725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a7f11ce41d44a6b12fb6040d580258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:38.183213 30413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a7f11ce41d44a6b12fb6040d580258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:38.186694 30828 tablet_bootstrap.cc:492] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Bootstrap starting.
I20260808 07:11:38.187067 30829 tablet_bootstrap.cc:492] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: Bootstrap starting.
I20260808 07:11:38.187429 30828 tablet_bootstrap.cc:654] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:38.187758 30829 tablet_bootstrap.cc:654] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:38.187789 30828 log.cc:824] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:38.188066 30829 log.cc:824] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:38.188696 30828 tablet_bootstrap.cc:492] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: No bootstrap required, opened a new log
I20260808 07:11:38.188891 30828 ts_tablet_manager.cc:1397] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:11:38.190451 30829 tablet_bootstrap.cc:492] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: No bootstrap required, opened a new log
I20260808 07:11:38.190521 30829 ts_tablet_manager.cc:1397] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:11:38.193105 30829 raft_consensus.cc:348] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.193234 30829 raft_consensus.cc:374] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:38.193256 30829 raft_consensus.cc:729] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97880488f60b4581b7f9fb132157599b, State: Initialized, Role: FOLLOWER
I20260808 07:11:38.193373 30829 consensus_queue.cc:260] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.193624 30829 ts_tablet_manager.cc:1428] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:38.193751 30485 heartbeater.cc:503] Master 127.22.2.190:41569 was elected leader, sending a full tablet report...
I20260808 07:11:38.193845 30830 tablet_bootstrap.cc:492] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: Bootstrap starting.
I20260808 07:11:38.194574 30830 tablet_bootstrap.cc:654] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:38.194859 30830 log.cc:824] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:38.195722 30830 tablet_bootstrap.cc:492] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: No bootstrap required, opened a new log
I20260808 07:11:38.195801 30830 ts_tablet_manager.cc:1397] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:38.196307 30828 raft_consensus.cc:348] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.196498 30828 raft_consensus.cc:374] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:38.196538 30828 raft_consensus.cc:729] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0296fb38106f45de9a3b43df9b814368, State: Initialized, Role: FOLLOWER
I20260808 07:11:38.196663 30828 consensus_queue.cc:260] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.197074 30644 heartbeater.cc:503] Master 127.22.2.190:41569 was elected leader, sending a full tablet report...
I20260808 07:11:38.197453 30828 ts_tablet_manager.cc:1428] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Time spent starting tablet: real 0.008s user 0.003s sys 0.001s
I20260808 07:11:38.198818 30830 raft_consensus.cc:348] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.198956 30830 raft_consensus.cc:374] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:38.199002 30830 raft_consensus.cc:729] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26a25f00305411abd569f0a54ea6cbf, State: Initialized, Role: FOLLOWER
I20260808 07:11:38.199106 30830 consensus_queue.cc:260] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.199373 30830 ts_tablet_manager.cc:1428] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:11:38.199443 30797 heartbeater.cc:503] Master 127.22.2.190:41569 was elected leader, sending a full tablet report...
W20260808 07:11:38.255268 30646 tablet.cc:2369] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:38.263207 30834 raft_consensus.cc:482] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:38.263337 30834 raft_consensus.cc:504] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.263659 30834 leader_election.cc:288] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215), d26a25f00305411abd569f0a54ea6cbf (127.22.2.131:41063)
I20260808 07:11:38.266718 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.264102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48777 (local address 127.22.2.131:41063)
0808 07:11:38.264169 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.264171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.264180 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:38.264206 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.264207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.264219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.264274 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:38.264618 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.265075 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.265959 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.266338 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.266394 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.266401 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.266600 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.266650 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:11:38.266709 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.263992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:41063 (local address 127.22.2.129:48777)
0808 07:11:38.264048 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.264057 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.264090 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.264439 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.264441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.264446 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:38.264530 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.264531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.265171 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.265172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.265867 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.265868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.266427 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.266428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.266515 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.266519 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.266576 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.266605 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":3}
I20260808 07:11:38.266770 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.263950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48395 (local address 127.22.2.130:38215)
0808 07:11:38.264015 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.264017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.264030 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:11:38.264060 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.264061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.264074 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.264170 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:38.264458 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.265018 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.265982 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.266406 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.266479 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.266486 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.266653 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.266694 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:11:38.266830 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.263861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:38215 (local address 127.22.2.129:48395)
0808 07:11:38.263920 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:38.263933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:38.263977 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:38.264246 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:38.264247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:38.264249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:38.264330 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.264332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.265105 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.265107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.265866 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:38.265869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.266494 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.266495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:38.266573 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.266577 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:38.266636 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:11:38.266672 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":2}
I20260808 07:11:38.267136 30591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a7f11ce41d44a6b12fb6040d580258" candidate_uuid: "97880488f60b4581b7f9fb132157599b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0296fb38106f45de9a3b43df9b814368" is_pre_election: true
I20260808 07:11:38.267228 30745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a7f11ce41d44a6b12fb6040d580258" candidate_uuid: "97880488f60b4581b7f9fb132157599b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26a25f00305411abd569f0a54ea6cbf" is_pre_election: true
I20260808 07:11:38.267298 30591 raft_consensus.cc:2454] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97880488f60b4581b7f9fb132157599b in term 0.
I20260808 07:11:38.267427 30745 raft_consensus.cc:2454] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97880488f60b4581b7f9fb132157599b in term 0.
I20260808 07:11:38.267585 30361 leader_election.cc:302] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0296fb38106f45de9a3b43df9b814368, 97880488f60b4581b7f9fb132157599b; no voters:
I20260808 07:11:38.267769 30834 raft_consensus.cc:2781] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:38.267844 30834 raft_consensus.cc:482] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:38.267876 30834 raft_consensus.cc:3037] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:38.268594 30834 raft_consensus.cc:504] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.268725 30834 leader_election.cc:288] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [CANDIDATE]: Term 1 election: Requested vote from peers 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215), d26a25f00305411abd569f0a54ea6cbf (127.22.2.131:41063)
I20260808 07:11:38.268890 30591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a7f11ce41d44a6b12fb6040d580258" candidate_uuid: "97880488f60b4581b7f9fb132157599b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0296fb38106f45de9a3b43df9b814368"
I20260808 07:11:38.268945 30745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a7f11ce41d44a6b12fb6040d580258" candidate_uuid: "97880488f60b4581b7f9fb132157599b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26a25f00305411abd569f0a54ea6cbf"
I20260808 07:11:38.268996 30591 raft_consensus.cc:3037] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:38.269032 30745 raft_consensus.cc:3037] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:38.269721 30745 raft_consensus.cc:2454] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97880488f60b4581b7f9fb132157599b in term 1.
I20260808 07:11:38.269788 30591 raft_consensus.cc:2454] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97880488f60b4581b7f9fb132157599b in term 1.
I20260808 07:11:38.269882 30361 leader_election.cc:302] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97880488f60b4581b7f9fb132157599b, d26a25f00305411abd569f0a54ea6cbf; no voters:
I20260808 07:11:38.270005 30834 raft_consensus.cc:2781] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:38.270193 30834 raft_consensus.cc:686] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 1 LEADER]: Becoming Leader. State: Replica: 97880488f60b4581b7f9fb132157599b, State: Running, Role: LEADER
I20260808 07:11:38.270399 30834 consensus_queue.cc:237] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } }
I20260808 07:11:38.271373 29397 catalog_manager.cc:5495] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b reported cstate change: term changed from 0 to 1, leader changed from <none> to 97880488f60b4581b7f9fb132157599b (127.22.2.129). New cstate: current_term: 1 leader_uuid: "97880488f60b4581b7f9fb132157599b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:38.279507 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.277256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53354 (local address 127.22.2.129:46071)
0808 07:11:38.277300 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.277302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.277323 (+ 21us) server_negotiation.cc:407] Connection header received
0808 07:11:38.277356 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.277357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.277374 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.277460 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:38.277570 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.278005 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.278754 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.278961 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.279046 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.279205 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:38.279299 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:38.279397 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.279416 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.279437 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:38.281122 30591 raft_consensus.cc:1261] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 1 FOLLOWER]: Refusing update from remote peer 97880488f60b4581b7f9fb132157599b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:38.281122 30745 raft_consensus.cc:1261] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 1 FOLLOWER]: Refusing update from remote peer 97880488f60b4581b7f9fb132157599b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:38.281486 30834 consensus_queue.cc:1035] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:38.281616 30839 consensus_queue.cc:1035] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:38.284008 30842 mvcc.cc:205] Tried to move back new op lower bound from 7316165010556928000 to 7316165010516193280. Current Snapshot: MvccSnapshot[applied={T|T < 7316165010556928000}]
I20260808 07:11:38.285084 30843 mvcc.cc:205] Tried to move back new op lower bound from 7316165010556928000 to 7316165010516193280. Current Snapshot: MvccSnapshot[applied={T|T < 7316165010556928000 or (T in {7316165010556928000})}]
I20260808 07:11:38.286755 29397 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:38.286855 29397 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:38.288081 29397 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ed2c8af33b24d7a9ae45e99e07787c5: Sending DeleteTablet for 3 replicas of tablet f4a7f11ce41d44a6b12fb6040d580258
I20260808 07:11:38.288539 30413 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a7f11ce41d44a6b12fb6040d580258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:38 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:53338
I20260808 07:11:38.288760 30851 tablet_replica.cc:330] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: stopping tablet replica
I20260808 07:11:38.288856 30851 raft_consensus.cc:2229] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:38.288947 30851 raft_consensus.cc:2258] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:38.288993 30569 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a7f11ce41d44a6b12fb6040d580258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:38 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:36146
I20260808 07:11:38.289070 30725 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a7f11ce41d44a6b12fb6040d580258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:38 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:51622
I20260808 07:11:38.289284 30853 tablet_replica.cc:330] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: stopping tablet replica
I20260808 07:11:38.289373 30853 raft_consensus.cc:2229] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:38.289438 30851 ts_tablet_manager.cc:1905] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:38.289515 30853 raft_consensus.cc:2258] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:38.289949 30853 ts_tablet_manager.cc:1905] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:38.290588 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 29360
I20260808 07:11:38.292212 30851 ts_tablet_manager.cc:1918] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:38.292296 30851 log.cc:1196] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/f4a7f11ce41d44a6b12fb6040d580258
I20260808 07:11:38.292670 30851 ts_tablet_manager.cc:1939] T f4a7f11ce41d44a6b12fb6040d580258 P 97880488f60b4581b7f9fb132157599b: Deleting consensus metadata
I20260808 07:11:38.292892 30852 tablet_replica.cc:330] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: stopping tablet replica
I20260808 07:11:38.293013 30852 raft_consensus.cc:2229] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:38.293152 30852 raft_consensus.cc:2258] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:38.293881 30852 ts_tablet_manager.cc:1905] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:38.295286 30853 ts_tablet_manager.cc:1918] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:38.295367 30853 log.cc:1196] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/f4a7f11ce41d44a6b12fb6040d580258
I20260808 07:11:38.295686 30853 ts_tablet_manager.cc:1939] T f4a7f11ce41d44a6b12fb6040d580258 P d26a25f00305411abd569f0a54ea6cbf: Deleting consensus metadata
I20260808 07:11:38.295854 30852 ts_tablet_manager.cc:1918] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:38.295948 30852 log.cc:1196] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/f4a7f11ce41d44a6b12fb6040d580258
I20260808 07:11:38.296319 30852 ts_tablet_manager.cc:1939] T f4a7f11ce41d44a6b12fb6040d580258 P 0296fb38106f45de9a3b43df9b814368: Deleting consensus metadata
W20260808 07:11:38.301798 30514 connection.cc:535] server connection from 127.0.0.1:36146 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:11:38.301944 30362 connection.cc:535] server connection from 127.0.0.1:53338 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:11:38.301988 30677 connection.cc:535] server connection from 127.0.0.1:51622 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:11:38.308344 30813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.289938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.22.2.188:45357)
0808 07:11:38.289982 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.289984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.299848 (+ 9864us) server_negotiation.cc:407] Connection header received
0808 07:11:38.299908 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.299909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.299926 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.299992 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.300859 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.301641 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.302903 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.303224 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.303297 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.305298 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.305300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.305300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.305308 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.307017 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.307410 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.307411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.307412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.307467 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.307718 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.307719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.307719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.307845 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.307918 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.308158 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.308243 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:38.311357 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.304015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.22.2.188:45357)
0808 07:11:38.304066 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.304068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.304256 (+ 188us) server_negotiation.cc:407] Connection header received
0808 07:11:38.304349 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.304350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.304361 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.304389 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.305128 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.305780 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.306633 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.306834 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.306910 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.308344 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.308346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.308346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.308354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.310152 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.310549 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.310550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.310550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.310605 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.310814 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.310815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.310816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.310926 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.310995 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.311188 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.311264 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:11:38.314888 30810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:38.304452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48340 (local address 127.22.2.189:37763)
0808 07:11:38.304503 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:38.304505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:38.307742 (+ 3237us) server_negotiation.cc:407] Connection header received
0808 07:11:38.307852 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:38.307853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:38.307870 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:38.307903 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:38.309104 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.309706 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.310506 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:38.310715 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:38.310791 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:38.312279 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:38.312280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:38.312281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:38.312288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:38.313708 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.314071 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.314072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.314072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.314130 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:38.314356 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:38.314357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:38.314358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:38.314467 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:38.314572 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:38.314676 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:11:38.314773 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:11:39.286357 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.286119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:36179)
0808 07:11:39.286237 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.286296 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":54,"threads_started":1}
I20260808 07:11:39.286330 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.285950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:46463)
0808 07:11:39.286182 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.286271 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":70,"threads_started":1}
W20260808 07:11:39.286559 30797 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:41569 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
W20260808 07:11:39.286748 30644 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:41569 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:39.286788 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.286467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:46703)
0808 07:11:39.286662 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.286741 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":61,"threads_started":1}
W20260808 07:11:39.287000 30485 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:41569 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:39.788902 30865 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9ed2c8af33b24d7a9ae45e99e07787c5)
I20260808 07:11:39.789001 30866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9ed2c8af33b24d7a9ae45e99e07787c5)
I20260808 07:11:39.789115 30865 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:39.789144 30866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:39.789455 30865 leader_election.cc:288] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569), 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189:37763)
I20260808 07:11:39.789670 30866 leader_election.cc:288] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569), b2a30a39bb284a39b15f828288bfd57b (127.22.2.188:45357)
I20260808 07:11:39.789892 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.789572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60440)
0808 07:11:39.789787 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.789841 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":144,"threads_started":1}
I20260808 07:11:39.789892 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.789574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60442)
0808 07:11:39.789792 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.789845 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":54,"threads_started":1}
W20260808 07:11:39.790009 30272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
W20260808 07:11:39.790077 30053 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:39.790488 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.790380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60456)
0808 07:11:39.790420 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.790443 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 07:11:39.790588 30053 leader_election.cc:334] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569): Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:39.790709 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.790380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60454)
0808 07:11:39.790661 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.790689 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266}
W20260808 07:11:39.791311 30272 leader_election.cc:334] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569): Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:39.793128 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.789601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37763 (local address 127.0.0.1:39012)
0808 07:11:39.790135 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.790145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:39.790220 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:39.790422 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:39.790423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:39.790424 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:39.790490 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:39.790492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:39.791471 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:39.791471 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:39.792290 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:39.792293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:39.792876 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:39.792877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:39.792945 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:39.792948 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:39.793015 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:11:39.793060 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":40,"threads_started":1}
I20260808 07:11:39.793123 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.789678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.22.2.189:37763)
0808 07:11:39.790008 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:39.790011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:39.790192 (+ 181us) server_negotiation.cc:407] Connection header received
0808 07:11:39.790293 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:39.790294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:39.790304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:39.790353 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:39.790571 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:39.791354 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:39.792398 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:39.792770 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:39.792873 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:39.792886 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:39.793020 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 07:11:39.793063 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":50,"threads_started":1}
I20260808 07:11:39.793226 30873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.789957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.22.2.188:45357)
0808 07:11:39.790069 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:39.790071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:39.790079 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:39.790280 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:39.790281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:39.790292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:39.790362 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:39.790620 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:39.791468 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:39.792439 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:39.792850 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:39.792953 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:39.792964 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:39.793113 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:11:39.793159 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":48,"threads_started":1}
I20260808 07:11:39.793231 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.789745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:45357 (local address 127.0.0.1:41648)
0808 07:11:39.789877 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.789888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:39.790103 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:39.790403 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:39.790404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:39.790406 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:39.790455 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:39.790457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:39.791550 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:39.791551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:39.792339 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:39.792341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:39.792958 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:39.792959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:39.793028 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:39.793031 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:39.793113 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:11:39.793165 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260808 07:11:39.793529 30079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2a30a39bb284a39b15f828288bfd57b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f3f91336b2a4abf9dbdbc4652149177" is_pre_election: true
I20260808 07:11:39.793555 30298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3f91336b2a4abf9dbdbc4652149177" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b2a30a39bb284a39b15f828288bfd57b" is_pre_election: true
I20260808 07:11:39.793644 30298 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3f91336b2a4abf9dbdbc4652149177 in term 1.
I20260808 07:11:39.793645 30079 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2a30a39bb284a39b15f828288bfd57b in term 1.
I20260808 07:11:39.793803 30271 leader_election.cc:302] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f3f91336b2a4abf9dbdbc4652149177, b2a30a39bb284a39b15f828288bfd57b; no voters: 9ed2c8af33b24d7a9ae45e99e07787c5
I20260808 07:11:39.793817 30054 leader_election.cc:302] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f3f91336b2a4abf9dbdbc4652149177, b2a30a39bb284a39b15f828288bfd57b; no voters: 9ed2c8af33b24d7a9ae45e99e07787c5
I20260808 07:11:39.793941 30866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 07:11:39.793999 30866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9ed2c8af33b24d7a9ae45e99e07787c5)
I20260808 07:11:39.794029 30866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:11:39.794227 30865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 07:11:39.794268 30865 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9ed2c8af33b24d7a9ae45e99e07787c5)
I20260808 07:11:39.794282 30865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:11:39.794791 30866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:39.794942 30866 leader_election.cc:288] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [CANDIDATE]: Term 2 election: Requested vote from peers 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569), b2a30a39bb284a39b15f828288bfd57b (127.22.2.188:45357)
I20260808 07:11:39.794981 30865 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:39.795130 30865 leader_election.cc:288] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 2 election: Requested vote from peers 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569), 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189:37763)
I20260808 07:11:39.795226 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.795052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60468)
0808 07:11:39.795150 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.795200 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 07:11:39.795228 30298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3f91336b2a4abf9dbdbc4652149177" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b2a30a39bb284a39b15f828288bfd57b"
I20260808 07:11:39.795250 30079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2a30a39bb284a39b15f828288bfd57b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f3f91336b2a4abf9dbdbc4652149177"
I20260808 07:11:39.795298 30298 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9f3f91336b2a4abf9dbdbc4652149177 in current term 2: Already voted for candidate b2a30a39bb284a39b15f828288bfd57b in this term.
I20260808 07:11:39.795347 30079 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b2a30a39bb284a39b15f828288bfd57b in current term 2: Already voted for candidate 9f3f91336b2a4abf9dbdbc4652149177 in this term.
I20260808 07:11:39.795372 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.795224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60478)
0808 07:11:39.795316 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.795344 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 07:11:39.795646 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.795576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60482)
0808 07:11:39.795608 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.795627 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 07:11:39.795723 30272 leader_election.cc:334] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569): Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:39.795780 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:39.795695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60496)
0808 07:11:39.795729 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:39.795755 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 07:11:39.795819 30271 leader_election.cc:302] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2a30a39bb284a39b15f828288bfd57b; no voters: 9ed2c8af33b24d7a9ae45e99e07787c5, 9f3f91336b2a4abf9dbdbc4652149177
W20260808 07:11:39.795879 30053 leader_election.cc:334] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569): Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:39.795897 30865 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 07:11:39.795924 30053 leader_election.cc:302] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f3f91336b2a4abf9dbdbc4652149177; no voters: 9ed2c8af33b24d7a9ae45e99e07787c5, b2a30a39bb284a39b15f828288bfd57b
I20260808 07:11:39.795991 30866 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 07:11:40.287321 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:40.286996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:37595)
0808 07:11:40.287167 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:40.287215 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":82,"threads_started":1}
I20260808 07:11:40.287405 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:40.287141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:35813)
0808 07:11:40.287313 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:40.287355 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260808 07:11:40.287845 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:40.287397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:59181)
0808 07:11:40.287751 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:40.287794 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":74,"threads_started":1}
I20260808 07:11:41.288120 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.287835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:42991)
0808 07:11:41.288008 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.288055 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
I20260808 07:11:41.288434 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.287798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:36495)
0808 07:11:41.287938 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.288055 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260808 07:11:41.288638 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.288293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:39095)
0808 07:11:41.288560 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.288596 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":80,"threads_started":1}
I20260808 07:11:41.325022 30882 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:41.325109 30882 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:41.325258 30882 leader_election.cc:288] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569), 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189:37763)
I20260808 07:11:41.325517 30079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2a30a39bb284a39b15f828288bfd57b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f3f91336b2a4abf9dbdbc4652149177" is_pre_election: true
I20260808 07:11:41.325568 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.325360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60698)
0808 07:11:41.325491 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.325527 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260808 07:11:41.325606 30079 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2a30a39bb284a39b15f828288bfd57b in term 2.
I20260808 07:11:41.325824 30271 leader_election.cc:302] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3f91336b2a4abf9dbdbc4652149177, b2a30a39bb284a39b15f828288bfd57b; no voters:
I20260808 07:11:41.325932 30882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 07:11:41.325982 30882 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:41.326042 30882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:11:41.326053 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.325929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60702)
0808 07:11:41.325966 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.325997 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 07:11:41.326120 30272 leader_election.cc:334] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569): Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:41.326825 30882 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:41.326963 30882 leader_election.cc:288] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 3 election: Requested vote from peers 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569), 9f3f91336b2a4abf9dbdbc4652149177 (127.22.2.189:37763)
I20260808 07:11:41.327123 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.327031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60704)
0808 07:11:41.327075 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.327099 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:11:41.327147 30079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2a30a39bb284a39b15f828288bfd57b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f3f91336b2a4abf9dbdbc4652149177"
I20260808 07:11:41.327261 30079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:11:41.327404 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.327313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60708)
0808 07:11:41.327360 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.327384 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:11:41.327481 30272 leader_election.cc:334] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569): Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
I20260808 07:11:41.327962 30079 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2a30a39bb284a39b15f828288bfd57b in term 3.
I20260808 07:11:41.328127 30271 leader_election.cc:302] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f3f91336b2a4abf9dbdbc4652149177, b2a30a39bb284a39b15f828288bfd57b; no voters: 9ed2c8af33b24d7a9ae45e99e07787c5
I20260808 07:11:41.328207 30882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:11:41.328308 30882 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [term 3 LEADER]: Becoming Leader. State: Replica: b2a30a39bb284a39b15f828288bfd57b, State: Running, Role: LEADER
I20260808 07:11:41.328363 30882 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } }
I20260808 07:11:41.328728 30885 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a30a39bb284a39b15f828288bfd57b. Latest consensus state: current_term: 3 leader_uuid: "b2a30a39bb284a39b15f828288bfd57b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:41.328843 30885 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:41.329025 30887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:41.329384 30887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:41.329437 30887 catalog_manager.cc:1259] Loaded cluster ID: ddfba8ff318b41969f8a285d353b4125
I20260808 07:11:41.329453 30887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:41.329612 30887 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:41.329721 30887 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Loaded TSK: 0
I20260808 07:11:41.329840 30887 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:11:41.403095 30287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:41.404917 30287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:41.406002 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.405893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60728)
0808 07:11:41.405935 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.405972 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:11:41.406025 30079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [term 3 FOLLOWER]: Refusing update from remote peer b2a30a39bb284a39b15f828288bfd57b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 07:11:41.406320 30882 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 07:11:41.406424 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.406312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.0.0.1:60736)
0808 07:11:41.406364 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.406405 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:11:41.406538 30272 consensus_peers.cc:487] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b -> Peer 9ed2c8af33b24d7a9ae45e99e07787c5 (127.22.2.190:41569): Couldn't send request to peer 9ed2c8af33b24d7a9ae45e99e07787c5. Status: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 07:11:41.407203 30889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a30a39bb284a39b15f828288bfd57b. Latest consensus state: current_term: 3 leader_uuid: "b2a30a39bb284a39b15f828288bfd57b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:41.407336 30889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:41.407923 30882 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b2a30a39bb284a39b15f828288bfd57b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:41.408015 30882 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:41.408109 30889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b2a30a39bb284a39b15f828288bfd57b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9ed2c8af33b24d7a9ae45e99e07787c5" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41569 } } peers { permanent_uuid: "9f3f91336b2a4abf9dbdbc4652149177" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37763 } attrs { promote: false } } peers { permanent_uuid: "b2a30a39bb284a39b15f828288bfd57b" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 45357 } attrs { promote: false } } }
I20260808 07:11:41.408241 30889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3f91336b2a4abf9dbdbc4652149177 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:41.413767 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.410247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.22.2.129:46071)
0808 07:11:41.410574 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:41.410577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:41.410587 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:41.410631 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:41.410632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:41.410644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:41.410909 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:41.410960 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.411503 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.412805 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.413416 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.413497 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.413511 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:41.413645 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 07:11:41.413695 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":82,"threads_started":1}
I20260808 07:11:41.413766 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.410115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46071 (local address 127.0.0.1:35412)
0808 07:11:41.410285 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.410299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:41.410357 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:41.410748 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:41.410749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:41.410751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:41.410807 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.410809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.411676 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.411677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.412700 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.412703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.413516 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.413517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.413578 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.413583 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:41.413635 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:11:41.413687 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":43,"thread_start_us":83,"threads_started":1}
I20260808 07:11:41.413945 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.410384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.22.2.130:38215)
0808 07:11:41.410492 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:41.410495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:41.410502 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:41.410527 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:41.410528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:41.410538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:41.410725 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:41.410767 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.411529 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.413024 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.413432 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.413557 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.413570 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:41.413809 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 07:11:41.413861 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":52,"threads_started":1}
I20260808 07:11:41.414052 30895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.410154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:38215 (local address 127.0.0.1:45432)
0808 07:11:41.410307 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.410317 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:41.410357 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:41.410631 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:41.410632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:41.410634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:41.410660 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.410662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.411678 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.411680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.412919 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.412921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.413565 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.413566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.413645 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.413650 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:41.413715 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:11:41.413767 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":79,"thread_start_us":89,"threads_started":1}
I20260808 07:11:41.414227 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.410120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.22.2.131:41063)
0808 07:11:41.410257 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:41.410259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:41.410414 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:11:41.410444 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:41.410446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:41.410458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:41.410527 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:41.410734 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.411525 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.413019 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.413671 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.413737 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.413750 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:41.414087 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 07:11:41.414141 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260808 07:11:41.414274 30897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.410034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:41063 (local address 127.0.0.1:48504)
0808 07:11:41.410281 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.410292 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:41.410355 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:41.410582 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:41.410583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:41.410585 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:41.410643 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.410644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.411710 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.411711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.412919 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.412922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.413785 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.413786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.413845 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.413848 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:41.413898 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:11:41.413943 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":32}
I20260808 07:11:41.414345 30569 tablet_service.cc:1467] Processing CreateTablet for tablet ff3b211bf3d5470ea53e5164008e0c50 (DEFAULT_TABLE table=test-table [id=970c3b8dd99b49f6b1afc98f4823c9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:41.414434 30413 tablet_service.cc:1467] Processing CreateTablet for tablet ff3b211bf3d5470ea53e5164008e0c50 (DEFAULT_TABLE table=test-table [id=970c3b8dd99b49f6b1afc98f4823c9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:41.414512 30569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff3b211bf3d5470ea53e5164008e0c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:41.414573 30413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff3b211bf3d5470ea53e5164008e0c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:41.415762 30902 tablet_bootstrap.cc:492] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b: Bootstrap starting.
I20260808 07:11:41.415953 30725 tablet_service.cc:1467] Processing CreateTablet for tablet ff3b211bf3d5470ea53e5164008e0c50 (DEFAULT_TABLE table=test-table [id=970c3b8dd99b49f6b1afc98f4823c9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:41.416103 30725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff3b211bf3d5470ea53e5164008e0c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:41.416069 30903 tablet_bootstrap.cc:492] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: Bootstrap starting.
I20260808 07:11:41.416296 30902 tablet_bootstrap.cc:654] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:41.416492 30903 tablet_bootstrap.cc:654] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:41.416847 30902 tablet_bootstrap.cc:492] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b: No bootstrap required, opened a new log
I20260808 07:11:41.416882 30902 ts_tablet_manager.cc:1397] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:41.416963 30902 raft_consensus.cc:348] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.417011 30902 raft_consensus.cc:374] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:41.417023 30902 raft_consensus.cc:729] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97880488f60b4581b7f9fb132157599b, State: Initialized, Role: FOLLOWER
I20260808 07:11:41.417088 30902 consensus_queue.cc:260] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.417275 30902 ts_tablet_manager.cc:1428] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:41.417579 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.417445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:57799)
0808 07:11:41.417512 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.417549 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:11:41.418058 30908 tablet_bootstrap.cc:492] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: Bootstrap starting.
I20260808 07:11:41.418182 30903 tablet_bootstrap.cc:492] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: No bootstrap required, opened a new log
I20260808 07:11:41.418263 30484 heartbeater.cc:503] Master 127.22.2.188:45357 was elected leader, sending a full tablet report...
I20260808 07:11:41.418298 30903 ts_tablet_manager.cc:1397] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:41.418387 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.418273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:54689)
0808 07:11:41.418321 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.418360 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:11:41.418467 30903 raft_consensus.cc:348] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.418533 30908 tablet_bootstrap.cc:654] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:41.418569 30903 raft_consensus.cc:374] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:41.418596 30903 raft_consensus.cc:729] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0296fb38106f45de9a3b43df9b814368, State: Initialized, Role: FOLLOWER
I20260808 07:11:41.418704 30903 consensus_queue.cc:260] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.418943 30903 ts_tablet_manager.cc:1428] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:41.419201 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.419072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:60171)
0808 07:11:41.419140 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.419174 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:11:41.419447 30643 heartbeater.cc:503] Master 127.22.2.188:45357 was elected leader, sending a full tablet report...
I20260808 07:11:41.419530 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.419401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:59857)
0808 07:11:41.419458 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.419500 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:11:41.419761 30908 tablet_bootstrap.cc:492] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: No bootstrap required, opened a new log
I20260808 07:11:41.419822 30908 ts_tablet_manager.cc:1397] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:41.419935 30908 raft_consensus.cc:348] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.419998 30908 raft_consensus.cc:374] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:41.420012 30908 raft_consensus.cc:729] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26a25f00305411abd569f0a54ea6cbf, State: Initialized, Role: FOLLOWER
I20260808 07:11:41.420075 30908 consensus_queue.cc:260] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.420285 30908 ts_tablet_manager.cc:1428] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:41.420559 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.420431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:49511)
0808 07:11:41.420491 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.420524 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 07:11:41.420776 30796 heartbeater.cc:503] Master 127.22.2.188:45357 was elected leader, sending a full tablet report...
I20260808 07:11:41.420945 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.420853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:53043)
0808 07:11:41.420892 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.420918 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 07:11:41.606566 30489 tablet.cc:2369] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:41.622850 30911 raft_consensus.cc:482] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:41.622953 30911 raft_consensus.cc:504] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.623270 30911 leader_election.cc:288] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215), 97880488f60b4581b7f9fb132157599b (127.22.2.129:46071)
I20260808 07:11:41.626196 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.623537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:37195 (local address 127.22.2.129:46071)
0808 07:11:41.623603 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:41.623605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:41.623617 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:11:41.623708 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:41.623708 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:41.623739 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:41.623825 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:41.624019 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.624505 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.625458 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.625841 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.625938 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.625949 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:41.626093 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 07:11:41.626132 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:11:41.626196 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.623459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46071 (local address 127.22.2.131:37195)
0808 07:11:41.623526 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.623539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:41.623627 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:41.623858 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:41.623859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:41.623862 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:41.623930 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.623934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.624594 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.624595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.625365 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.625367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.625939 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.625940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.626005 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.626009 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:41.626083 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:11:41.626124 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:11:41.626313 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.623529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:44583 (local address 127.22.2.130:38215)
0808 07:11:41.623603 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:41.623605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:41.623878 (+ 273us) server_negotiation.cc:407] Connection header received
0808 07:11:41.623983 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:41.623984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:41.623997 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:41.624044 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:41.624203 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.624685 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.625596 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.625953 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.626034 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.626044 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:41.626217 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 07:11:41.626255 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:11:41.626324 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.623459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:38215 (local address 127.22.2.131:44583)
0808 07:11:41.623787 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.623798 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:41.623913 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:41.624056 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:41.624057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:41.624058 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:41.624121 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.624125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.624774 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.624775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.625498 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:41.625499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.626055 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.626057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:41.626133 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.626136 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:41.626198 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:11:41.626238 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":122,"thread_start_us":132,"threads_started":1}
I20260808 07:11:41.626561 30434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff3b211bf3d5470ea53e5164008e0c50" candidate_uuid: "d26a25f00305411abd569f0a54ea6cbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97880488f60b4581b7f9fb132157599b" is_pre_election: true
I20260808 07:11:41.626704 30434 raft_consensus.cc:2454] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d26a25f00305411abd569f0a54ea6cbf in term 0.
I20260808 07:11:41.626672 30591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff3b211bf3d5470ea53e5164008e0c50" candidate_uuid: "d26a25f00305411abd569f0a54ea6cbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0296fb38106f45de9a3b43df9b814368" is_pre_election: true
I20260808 07:11:41.626785 30591 raft_consensus.cc:2454] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d26a25f00305411abd569f0a54ea6cbf in term 0.
I20260808 07:11:41.626873 30677 leader_election.cc:302] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97880488f60b4581b7f9fb132157599b, d26a25f00305411abd569f0a54ea6cbf; no voters:
I20260808 07:11:41.627061 30911 raft_consensus.cc:2781] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:41.627111 30911 raft_consensus.cc:482] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:41.627135 30911 raft_consensus.cc:3037] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:41.627924 30911 raft_consensus.cc:504] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.628063 30911 leader_election.cc:288] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [CANDIDATE]: Term 1 election: Requested vote from peers 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215), 97880488f60b4581b7f9fb132157599b (127.22.2.129:46071)
I20260808 07:11:41.628230 30434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff3b211bf3d5470ea53e5164008e0c50" candidate_uuid: "d26a25f00305411abd569f0a54ea6cbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97880488f60b4581b7f9fb132157599b"
I20260808 07:11:41.628306 30434 raft_consensus.cc:3037] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:41.628276 30591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff3b211bf3d5470ea53e5164008e0c50" candidate_uuid: "d26a25f00305411abd569f0a54ea6cbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0296fb38106f45de9a3b43df9b814368"
I20260808 07:11:41.628376 30591 raft_consensus.cc:3037] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:41.628958 30591 raft_consensus.cc:2454] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d26a25f00305411abd569f0a54ea6cbf in term 1.
I20260808 07:11:41.628988 30434 raft_consensus.cc:2454] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d26a25f00305411abd569f0a54ea6cbf in term 1.
I20260808 07:11:41.629153 30675 leader_election.cc:302] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0296fb38106f45de9a3b43df9b814368, d26a25f00305411abd569f0a54ea6cbf; no voters:
I20260808 07:11:41.629290 30911 raft_consensus.cc:2781] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:41.629463 30911 raft_consensus.cc:686] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 1 LEADER]: Becoming Leader. State: Replica: d26a25f00305411abd569f0a54ea6cbf, State: Running, Role: LEADER
I20260808 07:11:41.629571 30911 consensus_queue.cc:237] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } }
I20260808 07:11:41.629778 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.629679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:36595)
0808 07:11:41.629727 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.629753 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:11:41.630376 30287 catalog_manager.cc:5495] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf reported cstate change: term changed from 0 to 1, leader changed from <none> to d26a25f00305411abd569f0a54ea6cbf (127.22.2.131). New cstate: current_term: 1 leader_uuid: "d26a25f00305411abd569f0a54ea6cbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a25f00305411abd569f0a54ea6cbf" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:41.640614 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.638263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48518 (local address 127.22.2.131:41063)
0808 07:11:41.638322 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:41.638324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:41.638333 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:41.638381 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:41.638384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:41.638406 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:41.638483 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:41.638588 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.639025 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.639858 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:41.640043 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:41.640113 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:41.640295 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:41.640391 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:41.640466 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:41.640521 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 07:11:41.640540 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:41.641734 30591 raft_consensus.cc:1261] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 1 FOLLOWER]: Refusing update from remote peer d26a25f00305411abd569f0a54ea6cbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:41.641831 30434 raft_consensus.cc:1261] T ff3b211bf3d5470ea53e5164008e0c50 P 97880488f60b4581b7f9fb132157599b [term 1 FOLLOWER]: Refusing update from remote peer d26a25f00305411abd569f0a54ea6cbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:41.642081 30913 consensus_queue.cc:1035] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0296fb38106f45de9a3b43df9b814368" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:41.642249 30911 consensus_queue.cc:1035] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "97880488f60b4581b7f9fb132157599b" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:41.643590 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.643382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:50585)
0808 07:11:41.643499 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.643529 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 07:11:41.643903 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.643463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:49393)
0808 07:11:41.643836 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.643870 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353}
I20260808 07:11:41.644032 30916 mvcc.cc:205] Tried to move back new op lower bound from 7316165024322191360 to 7316165024275435520. Current Snapshot: MvccSnapshot[applied={T|T < 7316165024322191360}]
I20260808 07:11:41.645156 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.644900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.130:55393)
0808 07:11:41.645098 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.645126 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260808 07:11:41.645502 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.645363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.131:43339)
0808 07:11:41.645412 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.645469 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 07:11:41.646554 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.646444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41569 (local address 127.22.2.129:39995)
0808 07:11:41.646496 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.646523 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:11:41.646816 30286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:41.646906 30286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:41.648243 30286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2a30a39bb284a39b15f828288bfd57b: Sending DeleteTablet for 3 replicas of tablet ff3b211bf3d5470ea53e5164008e0c50
I20260808 07:11:41.648563 30413 tablet_service.cc:1514] Processing DeleteTablet for tablet ff3b211bf3d5470ea53e5164008e0c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:41 UTC) from {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:35412
I20260808 07:11:41.648680 30569 tablet_service.cc:1514] Processing DeleteTablet for tablet ff3b211bf3d5470ea53e5164008e0c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:41 UTC) from {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:45432
I20260808 07:11:41.648907 30725 tablet_service.cc:1514] Processing DeleteTablet for tablet ff3b211bf3d5470ea53e5164008e0c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:41 UTC) from {username='oryx', principal='oryx/127.22.2.188@KRBTEST.COM'} at 127.0.0.1:48504
I20260808 07:11:41.648975 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 30247
I20260808 07:11:41.651106 30928 tablet_replica.cc:330] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: stopping tablet replica
I20260808 07:11:41.651175 30928 raft_consensus.cc:2229] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:41.651278 30928 raft_consensus.cc:2258] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:41.651737 30928 ts_tablet_manager.cc:1905] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:41.653702 30928 ts_tablet_manager.cc:1918] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:41.653769 30928 log.cc:1196] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/ff3b211bf3d5470ea53e5164008e0c50
I20260808 07:11:41.654009 30928 ts_tablet_manager.cc:1939] T ff3b211bf3d5470ea53e5164008e0c50 P d26a25f00305411abd569f0a54ea6cbf: Deleting consensus metadata
I20260808 07:11:41.654465 30272 catalog_manager.cc:4841] TS d26a25f00305411abd569f0a54ea6cbf (127.22.2.131:41063): tablet ff3b211bf3d5470ea53e5164008e0c50 (table test-table [id=970c3b8dd99b49f6b1afc98f4823c9df]) successfully deleted
I20260808 07:11:41.657727 30927 tablet_replica.cc:330] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: stopping tablet replica
I20260808 07:11:41.657814 30927 raft_consensus.cc:2229] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:41.657877 30927 raft_consensus.cc:2258] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:41.658167 30927 ts_tablet_manager.cc:1905] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:41.659798 30927 ts_tablet_manager.cc:1918] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:41.659875 30927 log.cc:1196] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/ff3b211bf3d5470ea53e5164008e0c50
I20260808 07:11:41.660104 30927 ts_tablet_manager.cc:1939] T ff3b211bf3d5470ea53e5164008e0c50 P 0296fb38106f45de9a3b43df9b814368: Deleting consensus metadata
I20260808 07:11:41.660456 30272 catalog_manager.cc:4841] TS 0296fb38106f45de9a3b43df9b814368 (127.22.2.130:38215): tablet ff3b211bf3d5470ea53e5164008e0c50 (table test-table [id=970c3b8dd99b49f6b1afc98f4823c9df]) successfully deleted
I20260808 07:11:41.665702 30897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:41.665586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46071 (local address 127.0.0.1:35424)
0808 07:11:41.665640 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:41.665675 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:46071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:11:41.665891 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 30494
W20260808 07:11:41.665907 30273 catalog_manager.cc:4584] TS 97880488f60b4581b7f9fb132157599b (127.22.2.129:46071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff3b211bf3d5470ea53e5164008e0c50: Network error: Client connection negotiation failed: client connection to 127.22.2.129:46071: connect: Connection refused (error 111)
I20260808 07:11:41.674005 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 30650
I20260808 07:11:41.682039 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 29920
I20260808 07:11:41.688450 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 30132
2026-08-08T07:11:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6415 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:41 dist-test-slave-6k22 krb5kdc[30932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:41 dist-test-slave-6k22 krb5kdc[30932](info): set up 2 sockets
Aug 08 07:11:41 dist-test-slave-6k22 krb5kdc[30932](info): commencing operation
krb5kdc: starting...
W20260808 07:11:42.806346 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:42 dist-test-slave-6k22 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.190.keytab.
Entry for principal oryx/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.190.keytab.
I20260808 07:11:42.929672 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40679
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.190.keytab
--principal=oryx/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:43.010437 30948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:43.010689 30948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:43.010726 30948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:43.011899 30948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:43.011971 30948 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:11:43.011992 30948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:43.012013 30948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:43.012045 30948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:43.012068 30948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:43.013528 30948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40679
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.190.keytab
--principal=oryx/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:43.013825 30948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:43.014052 30948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:43.019127 30955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:43.019235 30956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:43.019438 30948 server_base.cc:1034] running on GCE node
W20260808 07:11:43.019207 30959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:43.019824 30948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:43.020076 30948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:43.021221 30948 hybrid_clock.cc:648] HybridClock initialized: now 1786173103021200 us; error 38 us; skew 500 ppm
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:43.023816 30948 init.cc:377] Logged in from keytab as oryx/127.22.2.190@KRBTEST.COM (short username oryx)
I20260808 07:11:43.024289 30948 webserver.cc:460] Webserver started at http://127.22.2.190:38467/ using document root <none> and password file <none>
I20260808 07:11:43.024518 30948 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:43.024586 30948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:43.024737 30948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:43.025816 30948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "585cd0c117524972993736839cc8d517"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "04deb0ea49e2a2fa259c42713eb35d64"
server_key_iv: "6782238bb04c57447d670c8c2baad30a"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.026204 30948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "585cd0c117524972993736839cc8d517"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "04deb0ea49e2a2fa259c42713eb35d64"
server_key_iv: "6782238bb04c57447d670c8c2baad30a"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.027527 30948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:43.028348 30968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.028551 30948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:43.028682 30948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "585cd0c117524972993736839cc8d517"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "04deb0ea49e2a2fa259c42713eb35d64"
server_key_iv: "6782238bb04c57447d670c8c2baad30a"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.028765 30948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:43.037345 30948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:43.039057 30948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:43.039222 30948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:43.044248 30948 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:40679
I20260808 07:11:43.044350 31041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:40679 every 8 connection(s)
I20260808 07:11:43.044704 30948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:43.045383 31042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:43.047801 31042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: Bootstrap starting.
I20260808 07:11:43.048403 31042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:43.048715 31042 log.cc:824] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:43.049412 31042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: No bootstrap required, opened a new log
I20260808 07:11:43.051435 31042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cd0c117524972993736839cc8d517" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40679 } }
I20260808 07:11:43.051621 31042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:43.051676 31042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 585cd0c117524972993736839cc8d517, State: Initialized, Role: FOLLOWER
I20260808 07:11:43.051838 31042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cd0c117524972993736839cc8d517" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40679 } }
I20260808 07:11:43.051920 31042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:43.051964 31042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:43.052011 31042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:43.052525 31042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cd0c117524972993736839cc8d517" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40679 } }
I20260808 07:11:43.052642 31042 leader_election.cc:302] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 585cd0c117524972993736839cc8d517; no voters:
I20260808 07:11:43.052820 31042 leader_election.cc:288] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:43.052856 31046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:43.053051 31046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [term 1 LEADER]: Becoming Leader. State: Replica: 585cd0c117524972993736839cc8d517, State: Running, Role: LEADER
I20260808 07:11:43.053085 31042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:43.053148 31046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cd0c117524972993736839cc8d517" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40679 } }
I20260808 07:11:43.053503 31047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "585cd0c117524972993736839cc8d517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cd0c117524972993736839cc8d517" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40679 } } }
I20260808 07:11:43.053606 31047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:43.053503 31048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 585cd0c117524972993736839cc8d517. Latest consensus state: current_term: 1 leader_uuid: "585cd0c117524972993736839cc8d517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cd0c117524972993736839cc8d517" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40679 } } }
I20260808 07:11:43.053784 31048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:43.054126 31053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:43.054661 31053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:43.054749 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 30948
I20260808 07:11:43.054857 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:43.055096 22538 external_mini_cluster.cc:1428] Setting key 2ef49ac063c888d00fb6685b1499774e
I20260808 07:11:43.060022 31053 catalog_manager.cc:1347] Generated new cluster ID: 96b5b2295acf413b9351ea66699359b4
I20260808 07:11:43.060076 31053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:43.067036 31053 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:43.067721 31053 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:43.069039 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.055538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.22.2.190:40679)
0808 07:11:43.055731 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.055740 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.055776 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:11:43.055893 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.055900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.055967 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.056076 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.056890 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.057930 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.058988 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.059412 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.059489 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.065525 (+ 6036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:43.065528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:43.065529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:43.065541 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:43.067476 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.067892 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.067894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.067894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.067975 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.068185 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.068186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.068186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.068409 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:43.068588 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.068666 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.068828 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260808 07:11:43.075937 31053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: Generated new TSK 0
I20260808 07:11:43.076157 31053 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
I20260808 07:11:43.110471 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40679
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:43.211652 31071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:43.211869 31071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:43.211897 31071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:43.213090 31071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:43.213142 31071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:43.213163 31071 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:43.213207 31071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:43.214753 31071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:43.215014 31071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:43.215272 31071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:43.220498 31076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:43.220498 31079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:43.220499 31077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:43.220851 31071 server_base.cc:1034] running on GCE node
I20260808 07:11:43.221117 31071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:43.221367 31071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:43.222537 31071 hybrid_clock.cc:648] HybridClock initialized: now 1786173103222522 us; error 28 us; skew 500 ppm
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:43.225171 31071 init.cc:377] Logged in from keytab as oryx/127.22.2.129@KRBTEST.COM (short username oryx)
I20260808 07:11:43.225696 31071 webserver.cc:460] Webserver started at http://127.22.2.129:41253/ using document root <none> and password file <none>
I20260808 07:11:43.225958 31071 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:43.226027 31071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:43.226176 31071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:43.227232 31071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "9e212e004bc94353bb3f931696d09d06"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "54f10c1480b31aa78ddec7c2c4281358"
server_key_iv: "b0b8e31766c67123de2494bca7be8440"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.227713 31071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "9e212e004bc94353bb3f931696d09d06"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "54f10c1480b31aa78ddec7c2c4281358"
server_key_iv: "b0b8e31766c67123de2494bca7be8440"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.229063 31071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:43.229789 31086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.229965 31071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:43.230068 31071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "9e212e004bc94353bb3f931696d09d06"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "54f10c1480b31aa78ddec7c2c4281358"
server_key_iv: "b0b8e31766c67123de2494bca7be8440"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.230151 31071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:43.241513 31071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:43.243481 31071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:43.243687 31071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:43.244374 31071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:43.244431 31071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.244500 31071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:43.244534 31071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:43.252343 31071 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:44803
I20260808 07:11:43.252452 31214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:44803 every 8 connection(s)
I20260808 07:11:43.252768 31071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:43.255645 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.244834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:41121 (local address 127.22.2.190:40679)
0808 07:11:43.244913 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.244916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.245267 (+ 351us) server_negotiation.cc:407] Connection header received
0808 07:11:43.245741 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.245743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.245758 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.245841 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.246588 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.247443 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.248550 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.248906 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.248978 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.252306 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:43.252312 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:43.252312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:43.252319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:43.254158 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.254571 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.254572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.254573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.254632 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.254930 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.254931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.254931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.255056 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:43.255133 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.255431 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.255525 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:11:43.255807 31095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.244888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40679 (local address 127.22.2.129:41121)
0808 07:11:43.245132 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.245182 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.245649 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.245984 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.245988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.246284 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:43.246470 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.246475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.247590 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.247591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.248477 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.248480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.249008 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.249009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.249123 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.249684 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:43.249696 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:43.252177 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:43.254241 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.254249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.254251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.254487 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.254699 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.254700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.254700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.254862 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.255133 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:43.255142 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:43.255281 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.255394 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.255557 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":86,"threads_started":1}
I20260808 07:11:43.256254 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 31071
I20260808 07:11:43.256378 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:43.256362 31215 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40679
I20260808 07:11:43.256440 31215 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:43.256568 22538 external_mini_cluster.cc:1428] Setting key 7edb263eaa99308da7f4ede8ee023972
I20260808 07:11:43.256625 31215 heartbeater.cc:511] Master 127.22.2.190:40679 requested a full tablet report, sending...
I20260808 07:11:43.257223 30988 ts_manager.cc:194] Registered new tserver with Master: 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803)
I20260808 07:11:43.257970 30988 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:41121
I20260808 07:11:43.265216 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.257495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:47437 (local address 127.22.2.190:40679)
0808 07:11:43.257534 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.257536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.259534 (+ 1998us) server_negotiation.cc:407] Connection header received
0808 07:11:43.259644 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.259645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.259656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.259713 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.260051 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.260524 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.261277 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.261483 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.261563 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.262579 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:43.262580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:43.262581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:43.262588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:43.264039 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.264402 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.264404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.264404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.264465 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.264674 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.264675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.264675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.264791 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:43.264884 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.265049 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.265119 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:11:43.265228 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.257500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40679 (local address 127.22.2.129:47437)
0808 07:11:43.259465 (+ 1965us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.259477 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.259586 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.259719 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.259720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.259909 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:43.259972 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.259973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.260621 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.260623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.261187 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.261189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.261640 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.261641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.261690 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.262172 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:43.262177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:43.262469 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:43.264128 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.264129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.264129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.264308 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.264537 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.264538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.264538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.264603 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.264874 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:43.264875 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:43.264923 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.265008 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.265121 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1934,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
I20260808 07:11:43.282714 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40679
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:43.364478 31226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:43.364719 31226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:43.364759 31226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:43.365922 31226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:43.365990 31226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:43.366024 31226 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:43.366067 31226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:43.367605 31226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:43.367892 31226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:43.368124 31226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:43.373334 31232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:43.373333 31231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:43.373396 31226 server_base.cc:1034] running on GCE node
W20260808 07:11:43.373378 31234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:43.373780 31226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:43.373991 31226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:43.375128 31226 hybrid_clock.cc:648] HybridClock initialized: now 1786173103375116 us; error 29 us; skew 500 ppm
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:43.377662 31226 init.cc:377] Logged in from keytab as oryx/127.22.2.130@KRBTEST.COM (short username oryx)
I20260808 07:11:43.378167 31226 webserver.cc:460] Webserver started at http://127.22.2.130:46771/ using document root <none> and password file <none>
I20260808 07:11:43.378423 31226 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:43.378494 31226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:43.378645 31226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:43.379760 31226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "6f95db4c74e04b11913c6f78fe8cbeff"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "18075fa1d995af1e19cc8dd616d139b1"
server_key_iv: "7c657c51cf0bbe0a84418674b3e550ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.380119 31226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "6f95db4c74e04b11913c6f78fe8cbeff"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "18075fa1d995af1e19cc8dd616d139b1"
server_key_iv: "7c657c51cf0bbe0a84418674b3e550ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.381457 31226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:43.382205 31241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.382382 31226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:43.382488 31226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "6f95db4c74e04b11913c6f78fe8cbeff"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "18075fa1d995af1e19cc8dd616d139b1"
server_key_iv: "7c657c51cf0bbe0a84418674b3e550ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.382560 31226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:43.393520 31226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:43.395896 31226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:43.396061 31226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:43.396621 31226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:43.396680 31226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.396723 31226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:43.396751 31226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:43.404381 31226 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:44361
I20260808 07:11:43.404471 31371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:44361 every 8 connection(s)
I20260808 07:11:43.404785 31226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:43.407114 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.397150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:47039 (local address 127.22.2.190:40679)
0808 07:11:43.397253 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.397256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.397498 (+ 242us) server_negotiation.cc:407] Connection header received
0808 07:11:43.398263 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.398264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.398275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.398318 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.399254 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.399924 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.401633 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.401853 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.401962 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.404556 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:43.404557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:43.404557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:43.404566 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:43.405958 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.406352 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.406352 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.406353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.406402 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.406618 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.406618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.406619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.406716 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:43.406764 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.406989 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.407051 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:11:43.407339 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.397119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40679 (local address 127.22.2.130:47039)
0808 07:11:43.397339 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.397392 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.398123 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.398421 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.398427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.398826 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:43.399134 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.399142 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.400000 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.400001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.401529 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.401533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.401982 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.401983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.402148 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.402674 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:43.402685 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:43.404448 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:43.406033 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.406040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.406041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.406284 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.406459 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.406459 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.406460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.406564 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.406770 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:43.406775 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:43.406881 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.406962 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.407077 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260808 07:11:43.407867 31372 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40679
I20260808 07:11:43.407972 31372 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:43.408159 31372 heartbeater.cc:511] Master 127.22.2.190:40679 requested a full tablet report, sending...
I20260808 07:11:43.408149 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 31226
I20260808 07:11:43.408279 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:43.408528 22538 external_mini_cluster.cc:1428] Setting key 322d758bf3bf853433e6a7fc3cfb139b
I20260808 07:11:43.408527 30987 ts_manager.cc:194] Registered new tserver with Master: 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361)
I20260808 07:11:43.409014 30987 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:47039
I20260808 07:11:43.415982 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.409241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40679 (local address 127.22.2.130:52455)
0808 07:11:43.409909 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.409922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.409988 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.410098 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.410099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.410314 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:43.410402 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.410404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.411072 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.411074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.411677 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.411678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.412171 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.412172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.412223 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.412619 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:43.412623 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:43.412927 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:43.414894 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.414894 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.414895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.415061 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.415272 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.415272 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.415273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.415327 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.415722 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:43.415722 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:43.415777 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.415834 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.415913 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":89,"threads_started":1}
I20260808 07:11:43.416066 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.409282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:52455 (local address 127.22.2.190:40679)
0808 07:11:43.409324 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.409326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.409994 (+ 668us) server_negotiation.cc:407] Connection header received
0808 07:11:43.410037 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.410037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.410045 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.410092 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.410470 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.410950 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.411768 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.412098 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.412167 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.413014 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:43.413015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:43.413016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:43.413023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:43.414800 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.415136 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.415137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.415137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.415202 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.415392 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.415393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.415394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.415641 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:43.415729 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.415888 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.415967 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for oryx/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
I20260808 07:11:43.434255 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40679
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:43.515178 31381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:43.515388 31381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:43.515413 31381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:43.516606 31381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:43.516652 31381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:43.516673 31381 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:43.516691 31381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:43.518095 31381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.2.190:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:43.518271 31381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:43.518471 31381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:43.523895 31387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:43.523913 31386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:43.523895 31389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:43.524201 31381 server_base.cc:1034] running on GCE node
I20260808 07:11:43.524467 31381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:43.524775 31381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:43.525929 31381 hybrid_clock.cc:648] HybridClock initialized: now 1786173103525920 us; error 29 us; skew 500 ppm
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:43.528710 31381 init.cc:377] Logged in from keytab as oryx/127.22.2.131@KRBTEST.COM (short username oryx)
I20260808 07:11:43.529088 31381 webserver.cc:460] Webserver started at http://127.22.2.131:34669/ using document root <none> and password file <none>
I20260808 07:11:43.529264 31381 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:43.529304 31381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:43.529392 31381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:43.530369 31381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "73e7c873eb34dc456f155a0b96ad7128"
server_key_iv: "1f7ab310b31fca32bc8442e6a60b4cc2"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.530674 31381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "73e7c873eb34dc456f155a0b96ad7128"
server_key_iv: "1f7ab310b31fca32bc8442e6a60b4cc2"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.532299 31381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:43.533200 31396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.533396 31381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:43.533496 31381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c"
format_stamp: "Formatted at 2026-08-08 07:11:43 on dist-test-slave-6k22"
server_key: "73e7c873eb34dc456f155a0b96ad7128"
server_key_iv: "1f7ab310b31fca32bc8442e6a60b4cc2"
server_key_version: "encryptionkey@0"
I20260808 07:11:43.533581 31381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:43.542243 31381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:43.544591 31381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:43.544756 31381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:43.545233 31381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:43.545297 31381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.545338 31381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:43.545372 31381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.552321 31381 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:35637
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:43.552404 31524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:35637 every 8 connection(s)
I20260808 07:11:43.552774 31381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:43.556185 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.545715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:40885 (local address 127.22.2.190:40679)
0808 07:11:43.545793 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.545795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.545978 (+ 183us) server_negotiation.cc:407] Connection header received
0808 07:11:43.546490 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.546492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.546504 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.546561 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.547424 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.548302 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.549686 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.550038 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.550150 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.552992 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:43.552994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:43.552995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:43.553004 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:43.554666 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.555264 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.555265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.555266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.555328 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.555591 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.555593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.555593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.555710 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:43.555765 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.556057 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.556121 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:11:43.556350 31408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.545695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40679 (local address 127.22.2.131:40885)
0808 07:11:43.545869 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.545907 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.546378 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.546628 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.546631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.547016 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:43.547296 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.547313 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.548406 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.548407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.549596 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.549599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.550159 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.550160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.550335 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.550877 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:43.550887 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:43.552867 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:43.554922 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.554930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.554932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.555179 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.555392 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.555393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.555393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.555496 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.555760 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:43.555766 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:43.555934 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.556026 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.556151 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260808 07:11:43.556855 31525 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40679
I20260808 07:11:43.556959 31525 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:43.557142 31525 heartbeater.cc:511] Master 127.22.2.190:40679 requested a full tablet report, sending...
I20260808 07:11:43.557600 30988 ts_manager.cc:194] Registered new tserver with Master: 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637)
I20260808 07:11:43.558116 30988 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:40885
I20260808 07:11:43.559278 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 31381
I20260808 07:11:43.559420 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:43.559666 22538 external_mini_cluster.cc:1428] Setting key 59cde259c11ef66f453f7021bc875b02
I20260808 07:11:43.560619 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:11:43.564006 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.557958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40679 (local address 127.22.2.131:34659)
0808 07:11:43.558087 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.558098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.558166 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.558294 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.558295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.558498 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:43.558553 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.558555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.559476 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.559477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.560331 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.560334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.560788 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.560789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.560859 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.561302 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:43.561313 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:43.561577 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:43.563105 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.563106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.563106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.563279 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.563475 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:43.563475 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:43.563475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:43.563517 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:43.563763 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:43.563764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:43.563802 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.563853 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.563933 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260808 07:11:43.564082 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.558028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:34659 (local address 127.22.2.190:40679)
0808 07:11:43.558074 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.558076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.558141 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:11:43.558225 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.558226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.558245 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.558291 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.558635 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.559346 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.560411 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.560712 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.560787 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.561675 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:43.561676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:43.561676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:43.561683 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:43.563023 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.563346 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.563347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.563347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.563410 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.563570 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.563570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.563571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.563701 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:43.563776 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.563879 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.563968 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:11:43.583204 31067 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:43.577677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.22.2.190:40679)
0808 07:11:43.577745 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.577747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.577758 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:43.577795 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.577796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.577813 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.577854 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.578482 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.578984 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.579970 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.580150 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.580202 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.583100 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41104: BlockingRecv error: recv got EOF from 127.0.0.1:41104 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 07:11:43 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:43.590839 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.584162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.22.2.190:40679)
0808 07:11:43.584231 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.584234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.584294 (+ 60us) server_negotiation.cc:407] Connection header received
0808 07:11:43.584339 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.584340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.584348 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.584395 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:43.584889 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.585376 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.586421 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.586639 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.586696 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.588781 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:43.588782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:43.588783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:43.588789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:43.589965 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.590255 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.590256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.590256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.590307 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:43.590461 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:43.590461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:43.590461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:43.590546 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:43.590611 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.590686 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.590775 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:11:43.592386 30988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:43.593940 30988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:43.600241 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.596186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44361 (local address 127.0.0.1:55572)
0808 07:11:43.596583 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.596595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.596650 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.596886 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.596889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.596892 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:43.596962 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.596964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.597936 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.597937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.598929 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.598932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.599940 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.599941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.600009 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.600018 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.600103 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.600157 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260808 07:11:43.600255 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.596389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.22.2.130:44361)
0808 07:11:43.596691 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.596696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.596706 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:43.596747 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.596749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.596766 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.596856 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:43.597029 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.597845 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.599027 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.599771 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.599832 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.599912 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.600119 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.600158 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
I20260808 07:11:43.600677 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.596186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35637 (local address 127.0.0.1:54902)
0808 07:11:43.596369 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.596412 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.596477 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.597170 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.597170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.597173 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:43.597235 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.597241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.598601 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.598602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.599491 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.599493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.600433 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.600434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.600492 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.600495 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.600552 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.600589 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":61,"threads_started":1}
I20260808 07:11:43.600709 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.596467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.22.2.131:35637)
0808 07:11:43.596993 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.596997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.597006 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:43.597044 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.597046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.597061 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.597303 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:43.597372 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.598464 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.599604 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.600314 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.600422 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.600496 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.600564 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.600607 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":89,"threads_started":1}
I20260808 07:11:43.601322 31298 tablet_service.cc:1467] Processing CreateTablet for tablet 2b41e1db95ef4cbbbcfe399ac773edab (DEFAULT_TABLE table=test-table [id=d3add1ede03146dbb8ed71f37633eb53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:43.601322 31453 tablet_service.cc:1467] Processing CreateTablet for tablet 2b41e1db95ef4cbbbcfe399ac773edab (DEFAULT_TABLE table=test-table [id=d3add1ede03146dbb8ed71f37633eb53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:43.601693 31298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b41e1db95ef4cbbbcfe399ac773edab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:43.601702 31453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b41e1db95ef4cbbbcfe399ac773edab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:43.603895 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.596475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.22.2.129:44803)
0808 07:11:43.596642 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.596648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.597043 (+ 395us) server_negotiation.cc:407] Connection header received
0808 07:11:43.597092 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.597094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.597110 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.597184 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:43.597404 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.598495 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.599818 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.600540 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.600606 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.600675 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.603714 (+ 3039us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.603770 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260808 07:11:43.604060 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.596199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:44803 (local address 127.0.0.1:59792)
0808 07:11:43.596779 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.596790 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.596843 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.597257 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.597259 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.597261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:43.597300 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.597314 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.598579 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.598580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.599728 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.599731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.600726 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.600727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.600797 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.600801 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.600856 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.600893 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":31,"thread_start_us":88,"threads_started":1}
I20260808 07:11:43.604791 31142 tablet_service.cc:1467] Processing CreateTablet for tablet 2b41e1db95ef4cbbbcfe399ac773edab (DEFAULT_TABLE table=test-table [id=d3add1ede03146dbb8ed71f37633eb53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:43.604941 31559 tablet_bootstrap.cc:492] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Bootstrap starting.
I20260808 07:11:43.604941 31558 tablet_bootstrap.cc:492] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Bootstrap starting.
I20260808 07:11:43.605149 31142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b41e1db95ef4cbbbcfe399ac773edab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:43.605626 31558 tablet_bootstrap.cc:654] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:43.605626 31559 tablet_bootstrap.cc:654] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:43.605983 31558 log.cc:824] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:43.605984 31559 log.cc:824] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:43.606782 31558 tablet_bootstrap.cc:492] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: No bootstrap required, opened a new log
I20260808 07:11:43.606861 31558 ts_tablet_manager.cc:1397] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:43.607076 31559 tablet_bootstrap.cc:492] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: No bootstrap required, opened a new log
I20260808 07:11:43.607148 31559 ts_tablet_manager.cc:1397] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:43.608652 31562 tablet_bootstrap.cc:492] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Bootstrap starting.
I20260808 07:11:43.609262 31562 tablet_bootstrap.cc:654] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:43.609267 31559 raft_consensus.cc:348] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.609267 31558 raft_consensus.cc:348] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.609437 31559 raft_consensus.cc:374] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:43.609437 31558 raft_consensus.cc:374] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:43.609474 31558 raft_consensus.cc:729] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a31cb83c8e4b0b8a8aaff5faedbc3c, State: Initialized, Role: FOLLOWER
I20260808 07:11:43.609474 31559 raft_consensus.cc:729] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f95db4c74e04b11913c6f78fe8cbeff, State: Initialized, Role: FOLLOWER
I20260808 07:11:43.609637 31562 log.cc:824] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:43.609620 31558 consensus_queue.cc:260] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.609620 31559 consensus_queue.cc:260] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.609917 31559 ts_tablet_manager.cc:1428] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:11:43.609917 31558 ts_tablet_manager.cc:1428] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:11:43.609947 31525 heartbeater.cc:503] Master 127.22.2.190:40679 was elected leader, sending a full tablet report...
I20260808 07:11:43.610038 31372 heartbeater.cc:503] Master 127.22.2.190:40679 was elected leader, sending a full tablet report...
I20260808 07:11:43.610502 31562 tablet_bootstrap.cc:492] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: No bootstrap required, opened a new log
I20260808 07:11:43.610592 31562 ts_tablet_manager.cc:1397] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:43.612370 31562 raft_consensus.cc:348] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.612511 31562 raft_consensus.cc:374] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:43.612558 31562 raft_consensus.cc:729] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e212e004bc94353bb3f931696d09d06, State: Initialized, Role: FOLLOWER
I20260808 07:11:43.612673 31562 consensus_queue.cc:260] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.612929 31562 ts_tablet_manager.cc:1428] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:11:43.612990 31215 heartbeater.cc:503] Master 127.22.2.190:40679 was elected leader, sending a full tablet report...
W20260808 07:11:43.655479 31373 tablet.cc:2369] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:43.753644 31216 tablet.cc:2369] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:43.803818 31526 tablet.cc:2369] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:43.912598 31565 raft_consensus.cc:482] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:43.912768 31565 raft_consensus.cc:504] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.913166 31565 leader_election.cc:288] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637), 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803)
I20260808 07:11:43.916651 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.913539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:36461 (local address 127.22.2.131:35637)
0808 07:11:43.913600 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.913602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.913619 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:11:43.913657 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.913658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.913670 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.913774 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:43.913982 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.914579 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.915695 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.916290 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.916383 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.916394 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.916561 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.916598 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:11:43.916652 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.913372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35637 (local address 127.22.2.130:36461)
0808 07:11:43.913515 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.913528 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.913591 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.913799 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.913801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.913803 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:43.913886 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.913888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.914645 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.914646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.915548 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.915550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.916392 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.916393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.916467 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.916472 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.916532 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.916572 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260808 07:11:43.917058 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b41e1db95ef4cbbbcfe399ac773edab" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" is_pre_election: true
I20260808 07:11:43.917233 31474 raft_consensus.cc:2454] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 0.
I20260808 07:11:43.917361 31569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.913514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:44803 (local address 127.22.2.130:34351)
0808 07:11:43.913708 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:43.913719 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:43.913784 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:43.913982 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:43.913983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:43.913985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:43.914057 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.914059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.915130 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.915131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.916320 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:43.916323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.917088 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.917089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:43.917171 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.917174 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:43.917244 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:11:43.917281 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":57,"threads_started":1}
I20260808 07:11:43.917399 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.913580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:34351 (local address 127.22.2.129:44803)
0808 07:11:43.913625 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.913627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.913786 (+ 159us) server_negotiation.cc:407] Connection header received
0808 07:11:43.913855 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.913856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.913870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.913924 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:43.914339 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.915023 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.916418 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.917005 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.917088 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.917096 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.917253 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.917298 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:11:43.917472 31245 leader_election.cc:302] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a31cb83c8e4b0b8a8aaff5faedbc3c, 6f95db4c74e04b11913c6f78fe8cbeff; no voters:
I20260808 07:11:43.917600 31565 raft_consensus.cc:2781] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:43.917665 31565 raft_consensus.cc:482] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:43.917698 31565 raft_consensus.cc:3037] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:43.917676 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b41e1db95ef4cbbbcfe399ac773edab" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e212e004bc94353bb3f931696d09d06" is_pre_election: true
I20260808 07:11:43.917821 31164 raft_consensus.cc:2454] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 0.
I20260808 07:11:43.918534 31565 raft_consensus.cc:504] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.918684 31565 leader_election.cc:288] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 election: Requested vote from peers 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637), 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803)
I20260808 07:11:43.918843 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b41e1db95ef4cbbbcfe399ac773edab" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c"
I20260808 07:11:43.918854 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b41e1db95ef4cbbbcfe399ac773edab" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e212e004bc94353bb3f931696d09d06"
I20260808 07:11:43.918944 31474 raft_consensus.cc:3037] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:43.918939 31164 raft_consensus.cc:3037] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:43.919689 31164 raft_consensus.cc:2454] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 1.
I20260808 07:11:43.919715 31474 raft_consensus.cc:2454] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 1.
I20260808 07:11:43.919878 31245 leader_election.cc:302] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a31cb83c8e4b0b8a8aaff5faedbc3c, 6f95db4c74e04b11913c6f78fe8cbeff; no voters:
I20260808 07:11:43.920020 31565 raft_consensus.cc:2781] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:43.920136 31565 raft_consensus.cc:686] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 LEADER]: Becoming Leader. State: Replica: 6f95db4c74e04b11913c6f78fe8cbeff, State: Running, Role: LEADER
I20260808 07:11:43.920236 31565 consensus_queue.cc:237] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:43.921029 30987 catalog_manager.cc:5495] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130). New cstate: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:43.952612 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:43.950352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55582 (local address 127.22.2.130:44361)
0808 07:11:43.950413 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:43.950414 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:43.950422 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:43.950483 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:43.950483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:43.950504 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:43.950568 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:43.950726 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.951190 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.951923 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:43.952105 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:43.952155 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:43.952302 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:43.952397 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:43.952451 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:43.952488 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:11:43.952504 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:11:43.954471 31474 raft_consensus.cc:1261] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Refusing update from remote peer 6f95db4c74e04b11913c6f78fe8cbeff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:43.954512 31164 raft_consensus.cc:1261] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Refusing update from remote peer 6f95db4c74e04b11913c6f78fe8cbeff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:43.954829 31570 consensus_queue.cc:1035] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Connected to new peer: Peer: permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:43.955015 31565 consensus_queue.cc:1035] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:43.957509 31575 mvcc.cc:205] Tried to move back new op lower bound from 7316165033794670592 to 7316165033658015744. Current Snapshot: MvccSnapshot[applied={T|T < 7316165033794670592}]
I20260808 07:11:43.960795 30988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:43.960897 30988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:43.961680 30988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: Sending DeleteTablet for 3 replicas of tablet 2b41e1db95ef4cbbbcfe399ac773edab
I20260808 07:11:43.962019 31453 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b41e1db95ef4cbbbcfe399ac773edab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:43 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:54902
I20260808 07:11:43.962095 31142 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b41e1db95ef4cbbbcfe399ac773edab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:43 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:59792
I20260808 07:11:43.962103 31298 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b41e1db95ef4cbbbcfe399ac773edab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:43 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:55572
I20260808 07:11:43.962328 31586 tablet_replica.cc:330] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: stopping tablet replica
I20260808 07:11:43.962466 31586 raft_consensus.cc:2229] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:43.962519 30988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 07:11:43.962607 31586 raft_consensus.cc:2258] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:43.962663 31584 tablet_replica.cc:330] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: stopping tablet replica
W20260808 07:11:43.962687 30988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:43.962752 31584 raft_consensus.cc:2229] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:43.962895 31584 raft_consensus.cc:2258] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:43.963450 31586 ts_tablet_manager.cc:1905] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:43.964082 31298 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d364eed1b43999dc522d9e06886c8 (DEFAULT_TABLE table=test-table [id=06a33618cc714d7dbcba9331b0c9dfb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:43.964082 31142 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d364eed1b43999dc522d9e06886c8 (DEFAULT_TABLE table=test-table [id=06a33618cc714d7dbcba9331b0c9dfb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:43.964208 31298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d364eed1b43999dc522d9e06886c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:43.964208 31142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d364eed1b43999dc522d9e06886c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:43.965476 31586 ts_tablet_manager.cc:1918] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:43.965497 31562 tablet_bootstrap.cc:492] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: Bootstrap starting.
I20260808 07:11:43.965526 31559 tablet_bootstrap.cc:492] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: Bootstrap starting.
I20260808 07:11:43.965678 31586 log.cc:1196] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/2b41e1db95ef4cbbbcfe399ac773edab
I20260808 07:11:43.965934 31562 tablet_bootstrap.cc:654] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:43.965994 31559 tablet_bootstrap.cc:654] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:43.966135 31586 ts_tablet_manager.cc:1939] T 2b41e1db95ef4cbbbcfe399ac773edab P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting consensus metadata
I20260808 07:11:43.966656 31559 tablet_bootstrap.cc:492] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: No bootstrap required, opened a new log
I20260808 07:11:43.966634 30970 catalog_manager.cc:4841] TS 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361): tablet 2b41e1db95ef4cbbbcfe399ac773edab (table test-table [id=d3add1ede03146dbb8ed71f37633eb53]) successfully deleted
I20260808 07:11:43.966677 31453 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d364eed1b43999dc522d9e06886c8 (DEFAULT_TABLE table=test-table [id=06a33618cc714d7dbcba9331b0c9dfb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:43.966701 31559 ts_tablet_manager.cc:1397] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:43.966792 31453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d364eed1b43999dc522d9e06886c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:43.966862 31559 raft_consensus.cc:348] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:43.966933 31559 raft_consensus.cc:374] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:43.966956 31559 raft_consensus.cc:729] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f95db4c74e04b11913c6f78fe8cbeff, State: Initialized, Role: FOLLOWER
I20260808 07:11:43.966941 31584 ts_tablet_manager.cc:1905] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:43.967000 31559 consensus_queue.cc:260] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:43.967089 31559 ts_tablet_manager.cc:1428] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:43.968185 31558 tablet_bootstrap.cc:492] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Bootstrap starting.
I20260808 07:11:43.968600 31558 tablet_bootstrap.cc:654] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:43.968782 31584 ts_tablet_manager.cc:1918] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:43.968819 31562 tablet_bootstrap.cc:492] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: No bootstrap required, opened a new log
I20260808 07:11:43.968863 31562 ts_tablet_manager.cc:1397] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:11:43.968963 31584 log.cc:1196] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/2b41e1db95ef4cbbbcfe399ac773edab
I20260808 07:11:43.969036 31562 raft_consensus.cc:348] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:43.969129 31562 raft_consensus.cc:374] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:43.969157 31562 raft_consensus.cc:729] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e212e004bc94353bb3f931696d09d06, State: Initialized, Role: FOLLOWER
I20260808 07:11:43.969210 31562 consensus_queue.cc:260] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:43.969285 31584 ts_tablet_manager.cc:1939] T 2b41e1db95ef4cbbbcfe399ac773edab P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting consensus metadata
I20260808 07:11:43.969359 31562 ts_tablet_manager.cc:1428] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:43.969694 30972 catalog_manager.cc:4841] TS 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637): tablet 2b41e1db95ef4cbbbcfe399ac773edab (table test-table [id=d3add1ede03146dbb8ed71f37633eb53]) successfully deleted
I20260808 07:11:43.971668 31585 tablet_replica.cc:330] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: stopping tablet replica
I20260808 07:11:43.971807 31585 raft_consensus.cc:2229] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:43.971931 31585 raft_consensus.cc:2258] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:43.971984 31558 tablet_bootstrap.cc:492] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: No bootstrap required, opened a new log
I20260808 07:11:43.972043 31558 ts_tablet_manager.cc:1397] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 07:11:43.972196 31558 raft_consensus.cc:348] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:43.972276 31558 raft_consensus.cc:374] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:43.972321 31558 raft_consensus.cc:729] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a31cb83c8e4b0b8a8aaff5faedbc3c, State: Initialized, Role: FOLLOWER
I20260808 07:11:43.972366 31558 consensus_queue.cc:260] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:43.972412 31585 ts_tablet_manager.cc:1905] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:43.972455 31558 ts_tablet_manager.cc:1428] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:43.973893 31585 ts_tablet_manager.cc:1918] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:43.973973 31585 log.cc:1196] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/2b41e1db95ef4cbbbcfe399ac773edab
I20260808 07:11:43.974539 31585 ts_tablet_manager.cc:1939] T 2b41e1db95ef4cbbbcfe399ac773edab P 9e212e004bc94353bb3f931696d09d06: Deleting consensus metadata
I20260808 07:11:43.974954 30969 catalog_manager.cc:4841] TS 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803): tablet 2b41e1db95ef4cbbbcfe399ac773edab (table test-table [id=d3add1ede03146dbb8ed71f37633eb53]) successfully deleted
I20260808 07:11:44.212165 31581 raft_consensus.cc:482] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:44.212280 31581 raft_consensus.cc:504] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.212509 31581 leader_election.cc:288] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803), 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637)
I20260808 07:11:44.212783 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d364eed1b43999dc522d9e06886c8" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" is_pre_election: true
I20260808 07:11:44.212785 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d364eed1b43999dc522d9e06886c8" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e212e004bc94353bb3f931696d09d06" is_pre_election: true
I20260808 07:11:44.212900 31474 raft_consensus.cc:2454] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 0.
I20260808 07:11:44.212900 31164 raft_consensus.cc:2454] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 0.
I20260808 07:11:44.213060 31242 leader_election.cc:302] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f95db4c74e04b11913c6f78fe8cbeff, 9e212e004bc94353bb3f931696d09d06; no voters:
I20260808 07:11:44.213182 31581 raft_consensus.cc:2781] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:44.213270 31581 raft_consensus.cc:482] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:44.213297 31581 raft_consensus.cc:3037] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.214118 31581 raft_consensus.cc:504] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.214272 31581 leader_election.cc:288] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 election: Requested vote from peers 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803), 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637)
I20260808 07:11:44.214466 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d364eed1b43999dc522d9e06886c8" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c"
I20260808 07:11:44.214466 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d364eed1b43999dc522d9e06886c8" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e212e004bc94353bb3f931696d09d06"
I20260808 07:11:44.214542 31474 raft_consensus.cc:3037] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.214542 31164 raft_consensus.cc:3037] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.215305 31164 raft_consensus.cc:2454] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 1.
I20260808 07:11:44.215313 31474 raft_consensus.cc:2454] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 1.
I20260808 07:11:44.215507 31245 leader_election.cc:302] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a31cb83c8e4b0b8a8aaff5faedbc3c, 6f95db4c74e04b11913c6f78fe8cbeff; no voters:
I20260808 07:11:44.215672 31581 raft_consensus.cc:2781] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:44.215746 31581 raft_consensus.cc:686] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 LEADER]: Becoming Leader. State: Replica: 6f95db4c74e04b11913c6f78fe8cbeff, State: Running, Role: LEADER
I20260808 07:11:44.215785 31581 consensus_queue.cc:237] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.216455 30987 catalog_manager.cc:5495] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130). New cstate: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:11:44 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173103, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.190@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:44.279838 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.272342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40679 (local address 127.0.0.1:41122)
0808 07:11:44.272514 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:44.272528 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:44.272604 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:44.272770 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:44.272771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:44.273070 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:44.273119 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.273122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.273813 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.273814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.274352 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.274354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.274775 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.274776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.274818 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.275396 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:44.275401 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:44.276860 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:44.278792 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:44.278798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:44.278800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:44.278987 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:44.279186 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:44.279187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:44.279187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:44.279260 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:44.279570 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:44.279576 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:44.279617 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 07:11:44.279669 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:11:44.279761 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260808 07:11:44.279907 31599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.272464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.22.2.190:40679)
0808 07:11:44.272637 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.272639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.272649 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:44.272671 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.272671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.272680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.272741 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:44.273215 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.273734 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.274443 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.274694 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.274761 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.276948 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:44.276950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:44.276950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:44.276956 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:44.278719 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:44.279064 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:44.279065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:44.279065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:44.279122 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:44.279360 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:44.279361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:44.279361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:44.279483 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:44.279606 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.279727 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.279817 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":147,"thread_start_us":109,"threads_started":1}
I20260808 07:11:44.281921 30987 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:41122:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:11:44.282101 30987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:44.283636 31453 tablet_service.cc:1467] Processing CreateTablet for tablet 20f1f6a0d9c646ff8636e8c5a71e943e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fdb4b4fafc97412ab7a6f68527a02766]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:44.283640 31298 tablet_service.cc:1467] Processing CreateTablet for tablet 20f1f6a0d9c646ff8636e8c5a71e943e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fdb4b4fafc97412ab7a6f68527a02766]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:44.283730 31142 tablet_service.cc:1467] Processing CreateTablet for tablet 20f1f6a0d9c646ff8636e8c5a71e943e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fdb4b4fafc97412ab7a6f68527a02766]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:44.283816 31453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f1f6a0d9c646ff8636e8c5a71e943e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.283818 31298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f1f6a0d9c646ff8636e8c5a71e943e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.283870 31142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f1f6a0d9c646ff8636e8c5a71e943e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.285416 31558 tablet_bootstrap.cc:492] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Bootstrap starting.
I20260808 07:11:44.285427 31562 tablet_bootstrap.cc:492] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: Bootstrap starting.
I20260808 07:11:44.285429 31559 tablet_bootstrap.cc:492] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: Bootstrap starting.
I20260808 07:11:44.285782 31559 tablet_bootstrap.cc:654] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.285782 31562 tablet_bootstrap.cc:654] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.285833 31558 tablet_bootstrap.cc:654] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.286387 31559 tablet_bootstrap.cc:492] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: No bootstrap required, opened a new log
I20260808 07:11:44.286433 31559 ts_tablet_manager.cc:1397] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:44.286569 31559 raft_consensus.cc:348] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.286639 31559 raft_consensus.cc:374] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.286625 31558 tablet_bootstrap.cc:492] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: No bootstrap required, opened a new log
I20260808 07:11:44.286654 31559 raft_consensus.cc:729] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f95db4c74e04b11913c6f78fe8cbeff, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.286672 31558 ts_tablet_manager.cc:1397] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:44.286684 31559 consensus_queue.cc:260] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.286777 31559 ts_tablet_manager.cc:1428] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:44.286798 31581 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.286826 31558 raft_consensus.cc:348] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.286871 31581 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:44.286912 31558 raft_consensus.cc:374] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.286945 31558 raft_consensus.cc:729] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a31cb83c8e4b0b8a8aaff5faedbc3c, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.286996 31558 consensus_queue.cc:260] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.287096 31558 ts_tablet_manager.cc:1428] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:44.287176 31562 tablet_bootstrap.cc:492] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: No bootstrap required, opened a new log
I20260808 07:11:44.287242 31562 ts_tablet_manager.cc:1397] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:44.287408 31562 raft_consensus.cc:348] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.287510 31562 raft_consensus.cc:374] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.287545 31562 raft_consensus.cc:729] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e212e004bc94353bb3f931696d09d06, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.287621 31562 consensus_queue.cc:260] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.287672 31563 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.287781 31562 ts_tablet_manager.cc:1428] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:44.287809 31563 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:44.287914 31567 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.287998 31567 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:44.401451 31581 raft_consensus.cc:482] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:44.401551 31581 raft_consensus.cc:504] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.401722 31581 leader_election.cc:288] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637), 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803)
I20260808 07:11:44.402030 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f1f6a0d9c646ff8636e8c5a71e943e" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" is_pre_election: true
I20260808 07:11:44.402030 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f1f6a0d9c646ff8636e8c5a71e943e" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e212e004bc94353bb3f931696d09d06" is_pre_election: true
I20260808 07:11:44.402153 31164 raft_consensus.cc:2454] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 0.
I20260808 07:11:44.402153 31474 raft_consensus.cc:2454] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 0.
I20260808 07:11:44.402302 31245 leader_election.cc:302] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a31cb83c8e4b0b8a8aaff5faedbc3c, 6f95db4c74e04b11913c6f78fe8cbeff; no voters:
I20260808 07:11:44.402419 31581 raft_consensus.cc:2781] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:44.402467 31581 raft_consensus.cc:482] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:44.402493 31581 raft_consensus.cc:3037] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.403180 31581 raft_consensus.cc:504] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.403337 31581 leader_election.cc:288] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 election: Requested vote from peers 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637), 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803)
I20260808 07:11:44.403491 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f1f6a0d9c646ff8636e8c5a71e943e" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c"
I20260808 07:11:44.403491 31164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f1f6a0d9c646ff8636e8c5a71e943e" candidate_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e212e004bc94353bb3f931696d09d06"
I20260808 07:11:44.403595 31164 raft_consensus.cc:3037] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.403595 31474 raft_consensus.cc:3037] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.404388 31474 raft_consensus.cc:2454] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 1.
I20260808 07:11:44.404388 31164 raft_consensus.cc:2454] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f95db4c74e04b11913c6f78fe8cbeff in term 1.
I20260808 07:11:44.404549 31245 leader_election.cc:302] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a31cb83c8e4b0b8a8aaff5faedbc3c, 6f95db4c74e04b11913c6f78fe8cbeff; no voters:
I20260808 07:11:44.404693 31581 raft_consensus.cc:2781] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:44.404762 31581 raft_consensus.cc:686] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 LEADER]: Becoming Leader. State: Replica: 6f95db4c74e04b11913c6f78fe8cbeff, State: Running, Role: LEADER
I20260808 07:11:44.404820 31581 consensus_queue.cc:237] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } }
I20260808 07:11:44.405118 31570 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: TxnStatusTablet state changed. Reason: New leader 6f95db4c74e04b11913c6f78fe8cbeff. Latest consensus state: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.405216 31570 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:44.405452 31604 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:11:44.405441 30987 catalog_manager.cc:5495] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130). New cstate: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:44.405566 31604 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:11:44.432950 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.430307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44361 (local address 127.0.0.1:55590)
0808 07:11:44.430381 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:44.430393 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:44.430448 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:44.430588 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:44.430589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:44.430590 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:44.430647 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.430649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.431356 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.431357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.432095 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.432098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.432480 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.432481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.432538 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.432556 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.432765 (+ 209us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.432774 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:11:44.432818 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 07:11:44.432884 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:11:44.432945 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.430392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.22.2.130:44361)
0808 07:11:44.430464 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.430466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.430491 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:11:44.430520 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.430521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.430531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.430573 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:44.430776 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.431269 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.432184 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.432393 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.432468 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.432647 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.432683 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.432750 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.432838 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.432859 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:11:44.434005 31474 raft_consensus.cc:1261] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Refusing update from remote peer 6f95db4c74e04b11913c6f78fe8cbeff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:44.434006 31164 raft_consensus.cc:1261] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Refusing update from remote peer 6f95db4c74e04b11913c6f78fe8cbeff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:44.434248 31570 consensus_queue.cc:1035] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Connected to new peer: Peer: permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:44.434350 31581 consensus_queue.cc:1035] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:44.435400 31567 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: TxnStatusTablet state changed. Reason: New leader 6f95db4c74e04b11913c6f78fe8cbeff. Latest consensus state: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.435431 31565 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.435503 31567 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:44.435518 31565 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:44.435598 31573 mvcc.cc:205] Tried to move back new op lower bound from 7316165035759939584 to 7316165035642527744. Current Snapshot: MvccSnapshot[applied={T|T < 7316165035759939584}]
I20260808 07:11:44.435701 31565 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.435771 31565 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 6f95db4c74e04b11913c6f78fe8cbeff: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:44.436048 31574 mvcc.cc:205] Tried to move back new op lower bound from 7316165035759939584 to 7316165035642527744. Current Snapshot: MvccSnapshot[applied={T|T < 7316165035759939584}]
I20260808 07:11:44.436345 31609 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.436441 31609 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:44.436620 31567 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.436702 31567 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 9e212e004bc94353bb3f931696d09d06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:44.437448 31563 tablet_replica.cc:439] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: TxnStatusTablet state changed. Reason: New leader 6f95db4c74e04b11913c6f78fe8cbeff. Latest consensus state: current_term: 1 leader_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } } peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } }
I20260808 07:11:44.437541 31563 tablet_replica.cc:442] T 20f1f6a0d9c646ff8636e8c5a71e943e P 22a31cb83c8e4b0b8a8aaff5faedbc3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:44.441855 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.439282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:37459 (local address 127.22.2.130:44361)
0808 07:11:44.439330 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.439332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.439481 (+ 149us) server_negotiation.cc:407] Connection header received
0808 07:11:44.439532 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.439533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.439541 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.439631 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:44.439746 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.440175 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.441265 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.441432 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.441482 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.441661 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.441684 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.441720 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.441792 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.441806 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:11:44.441951 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.439199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44361 (local address 127.22.2.130:37459)
0808 07:11:44.439406 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:44.439416 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:44.439477 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:44.439636 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:44.439637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:44.439638 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:44.439687 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.439689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.440241 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.440242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.441168 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.441170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.441500 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.441501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.441562 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.441580 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.441735 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.441737 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:44.441791 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:11:44.441840 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":107,"threads_started":1}
I20260808 07:11:44.443893 31164 raft_consensus.cc:1261] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Refusing update from remote peer 6f95db4c74e04b11913c6f78fe8cbeff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:44.443890 31474 raft_consensus.cc:1261] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Refusing update from remote peer 6f95db4c74e04b11913c6f78fe8cbeff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:44.444077 31570 consensus_queue.cc:1035] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:44.444185 31581 consensus_queue.cc:1035] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [LEADER]: Connected to new peer: Peer: permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:44.457507 30988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:44.457612 30988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:44.458241 30988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: Sending DeleteTablet for 3 replicas of tablet 0e1d364eed1b43999dc522d9e06886c8
I20260808 07:11:44.458564 31142 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d364eed1b43999dc522d9e06886c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:59792
I20260808 07:11:44.458561 31297 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d364eed1b43999dc522d9e06886c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:55572
I20260808 07:11:44.458561 31453 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d364eed1b43999dc522d9e06886c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:54902
I20260808 07:11:44.458659 31585 tablet_replica.cc:330] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: stopping tablet replica
I20260808 07:11:44.458659 31586 tablet_replica.cc:330] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: stopping tablet replica
I20260808 07:11:44.458668 31584 tablet_replica.cc:330] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: stopping tablet replica
I20260808 07:11:44.458714 31585 raft_consensus.cc:2229] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:44.458724 31584 raft_consensus.cc:2229] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:44.458725 31586 raft_consensus.cc:2229] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:44.458751 31585 raft_consensus.cc:2258] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.458760 31584 raft_consensus.cc:2258] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.458781 31586 raft_consensus.cc:2258] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.459054 31586 ts_tablet_manager.cc:1905] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:44.459137 31585 ts_tablet_manager.cc:1905] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:44.459136 31584 ts_tablet_manager.cc:1905] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:44.460934 31585 ts_tablet_manager.cc:1918] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:44.460989 31585 log.cc:1196] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/0e1d364eed1b43999dc522d9e06886c8
I20260808 07:11:44.460999 31586 ts_tablet_manager.cc:1918] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:44.461059 31586 log.cc:1196] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/0e1d364eed1b43999dc522d9e06886c8
I20260808 07:11:44.461218 31585 ts_tablet_manager.cc:1939] T 0e1d364eed1b43999dc522d9e06886c8 P 9e212e004bc94353bb3f931696d09d06: Deleting consensus metadata
I20260808 07:11:44.461318 31586 ts_tablet_manager.cc:1939] T 0e1d364eed1b43999dc522d9e06886c8 P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting consensus metadata
I20260808 07:11:44.461514 30969 catalog_manager.cc:4841] TS 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803): tablet 0e1d364eed1b43999dc522d9e06886c8 (table test-table [id=06a33618cc714d7dbcba9331b0c9dfb4]) successfully deleted
I20260808 07:11:44.461688 30970 catalog_manager.cc:4841] TS 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361): tablet 0e1d364eed1b43999dc522d9e06886c8 (table test-table [id=06a33618cc714d7dbcba9331b0c9dfb4]) successfully deleted
I20260808 07:11:44.462565 31584 ts_tablet_manager.cc:1918] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:44.462643 31584 log.cc:1196] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/0e1d364eed1b43999dc522d9e06886c8
I20260808 07:11:44.462857 31584 ts_tablet_manager.cc:1939] T 0e1d364eed1b43999dc522d9e06886c8 P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting consensus metadata
I20260808 07:11:44.463188 30972 catalog_manager.cc:4841] TS 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637): tablet 0e1d364eed1b43999dc522d9e06886c8 (table test-table [id=06a33618cc714d7dbcba9331b0c9dfb4]) successfully deleted
Aug 08 07:11:44 dist-test-slave-6k22 krb5kdc[30932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:11:44 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:44 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:11:44.486141 31599 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:44.480195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.22.2.190:40679)
0808 07:11:44.480264 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.480266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.480373 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:11:44.480441 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.480443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.480458 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.480517 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:44.481108 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.481880 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.482634 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.482942 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.483022 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.486073 (+ 3051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41136: BlockingRecv error: recv got EOF from 127.0.0.1:41136 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 07:11:44 dist-test-slave-6k22 krb5kdc[30932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM
I20260808 07:11:44.493894 31599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.486953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.22.2.190:40679)
0808 07:11:44.486995 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.486997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.487037 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:11:44.487116 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.487117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.487125 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.487198 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:44.487737 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.488203 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.488900 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.489094 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.489169 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.491262 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:44.491264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:44.491264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:44.491271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:44.492739 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:44.493092 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:44.493093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:44.493093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:44.493156 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:44.493374 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:44.493375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:44.493376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:44.493488 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:44.493595 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.493702 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.493786 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:11:44.495772 30988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:44.495967 30988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:44.497396 31142 tablet_service.cc:1467] Processing CreateTablet for tablet a58afc299377437dac9a8670b297de4c (DEFAULT_TABLE table=test-table [id=ba0ebba7342949d7bc1ea2ab94ee1510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:44.497433 31297 tablet_service.cc:1467] Processing CreateTablet for tablet a58afc299377437dac9a8670b297de4c (DEFAULT_TABLE table=test-table [id=ba0ebba7342949d7bc1ea2ab94ee1510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:44.497546 31142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58afc299377437dac9a8670b297de4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.497581 31297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58afc299377437dac9a8670b297de4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.498770 31559 tablet_bootstrap.cc:492] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: Bootstrap starting.
I20260808 07:11:44.498867 31453 tablet_service.cc:1467] Processing CreateTablet for tablet a58afc299377437dac9a8670b297de4c (DEFAULT_TABLE table=test-table [id=ba0ebba7342949d7bc1ea2ab94ee1510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:44.498989 31453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58afc299377437dac9a8670b297de4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.498998 31562 tablet_bootstrap.cc:492] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: Bootstrap starting.
I20260808 07:11:44.499099 31559 tablet_bootstrap.cc:654] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.499423 31562 tablet_bootstrap.cc:654] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.500212 31562 tablet_bootstrap.cc:492] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: No bootstrap required, opened a new log
I20260808 07:11:44.500272 31562 ts_tablet_manager.cc:1397] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:44.500448 31558 tablet_bootstrap.cc:492] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Bootstrap starting.
I20260808 07:11:44.500432 31562 raft_consensus.cc:348] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.500500 31562 raft_consensus.cc:374] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.500523 31562 raft_consensus.cc:729] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e212e004bc94353bb3f931696d09d06, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.500567 31562 consensus_queue.cc:260] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.500653 31562 ts_tablet_manager.cc:1428] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:44.500870 31558 tablet_bootstrap.cc:654] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.500979 31559 tablet_bootstrap.cc:492] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: No bootstrap required, opened a new log
I20260808 07:11:44.501027 31559 ts_tablet_manager.cc:1397] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:11:44.501151 31559 raft_consensus.cc:348] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.501232 31559 raft_consensus.cc:374] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.501257 31559 raft_consensus.cc:729] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f95db4c74e04b11913c6f78fe8cbeff, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.501295 31559 consensus_queue.cc:260] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.501370 31559 ts_tablet_manager.cc:1428] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:44.501520 31558 tablet_bootstrap.cc:492] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: No bootstrap required, opened a new log
I20260808 07:11:44.501571 31558 ts_tablet_manager.cc:1397] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:44.501713 31558 raft_consensus.cc:348] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.501781 31558 raft_consensus.cc:374] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.501816 31558 raft_consensus.cc:729] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a31cb83c8e4b0b8a8aaff5faedbc3c, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.501858 31558 consensus_queue.cc:260] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.501952 31558 ts_tablet_manager.cc:1428] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:44.555219 31567 raft_consensus.cc:482] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:44.555330 31567 raft_consensus.cc:504] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.555783 31567 leader_election.cc:288] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361), 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637)
I20260808 07:11:44.560335 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.556226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:45353 (local address 127.22.2.130:44361)
0808 07:11:44.556314 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.556317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.556327 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:44.556357 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.556358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.556368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.556429 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:44.556783 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.557492 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.559176 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.559870 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.559959 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.559972 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.560214 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.560262 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:11:44.560360 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.555985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44361 (local address 127.22.2.129:45353)
0808 07:11:44.556171 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:44.556185 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:44.556252 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:44.556501 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:44.556502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:44.556505 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:44.556704 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.556708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.557614 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.557616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.559069 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.559072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.560014 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.560015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.560102 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.560107 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:44.560199 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:11:44.560250 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":59,"thread_start_us":108,"threads_started":1}
I20260808 07:11:44.560493 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.556060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48667 (local address 127.22.2.131:35637)
0808 07:11:44.556174 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.556176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.556329 (+ 153us) server_negotiation.cc:407] Connection header received
0808 07:11:44.556386 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.556387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.556396 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.556476 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:44.556688 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.557493 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.559361 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.560062 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.560150 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.560161 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.560367 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.560412 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":49,"threads_started":1}
I20260808 07:11:44.560489 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.556007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:35637 (local address 127.22.2.129:48667)
0808 07:11:44.556218 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:44.556229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:44.556326 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:44.556497 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:44.556498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:44.556501 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:44.556620 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.556623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.557622 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.557623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.559271 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.559273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.560177 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.560179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.560243 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.560247 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:44.560358 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:11:44.560401 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":64,"thread_start_us":143,"threads_started":1}
I20260808 07:11:44.561141 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58afc299377437dac9a8670b297de4c" candidate_uuid: "9e212e004bc94353bb3f931696d09d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" is_pre_election: true
I20260808 07:11:44.561149 31321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58afc299377437dac9a8670b297de4c" candidate_uuid: "9e212e004bc94353bb3f931696d09d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" is_pre_election: true
I20260808 07:11:44.561251 31474 raft_consensus.cc:2454] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e212e004bc94353bb3f931696d09d06 in term 0.
I20260808 07:11:44.561298 31321 raft_consensus.cc:2454] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e212e004bc94353bb3f931696d09d06 in term 0.
I20260808 07:11:44.561461 31090 leader_election.cc:302] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a31cb83c8e4b0b8a8aaff5faedbc3c, 9e212e004bc94353bb3f931696d09d06; no voters:
I20260808 07:11:44.561654 31567 raft_consensus.cc:2781] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:44.561707 31567 raft_consensus.cc:482] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:44.561733 31567 raft_consensus.cc:3037] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.562419 31567 raft_consensus.cc:504] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.562580 31567 leader_election.cc:288] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [CANDIDATE]: Term 1 election: Requested vote from peers 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361), 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637)
I20260808 07:11:44.562749 31321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58afc299377437dac9a8670b297de4c" candidate_uuid: "9e212e004bc94353bb3f931696d09d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f95db4c74e04b11913c6f78fe8cbeff"
I20260808 07:11:44.562747 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58afc299377437dac9a8670b297de4c" candidate_uuid: "9e212e004bc94353bb3f931696d09d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c"
I20260808 07:11:44.562834 31321 raft_consensus.cc:3037] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.562855 31474 raft_consensus.cc:3037] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.563493 31321 raft_consensus.cc:2454] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e212e004bc94353bb3f931696d09d06 in term 1.
I20260808 07:11:44.563558 31474 raft_consensus.cc:2454] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e212e004bc94353bb3f931696d09d06 in term 1.
I20260808 07:11:44.563714 31088 leader_election.cc:302] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f95db4c74e04b11913c6f78fe8cbeff, 9e212e004bc94353bb3f931696d09d06; no voters:
I20260808 07:11:44.563858 31567 raft_consensus.cc:2781] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:44.564087 31567 raft_consensus.cc:686] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 1 LEADER]: Becoming Leader. State: Replica: 9e212e004bc94353bb3f931696d09d06, State: Running, Role: LEADER
I20260808 07:11:44.564181 31567 consensus_queue.cc:237] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.565038 30988 catalog_manager.cc:5495] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e212e004bc94353bb3f931696d09d06 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "9e212e004bc94353bb3f931696d09d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:44.575850 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.573425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.22.2.129:44803)
0808 07:11:44.573599 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.573600 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.573607 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:44.573642 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.573642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.573659 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.573735 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:44.573840 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.574328 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.575023 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.575236 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.575306 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.575455 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.575605 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.575678 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.575713 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.575731 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
I20260808 07:11:44.576891 31474 raft_consensus.cc:1261] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Refusing update from remote peer 9e212e004bc94353bb3f931696d09d06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:44.576944 31321 raft_consensus.cc:1261] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Refusing update from remote peer 9e212e004bc94353bb3f931696d09d06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:44.577184 31567 consensus_queue.cc:1035] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:44.577332 31637 consensus_queue.cc:1035] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:44.580703 30987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:44.580809 30987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:44.581444 30987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: Sending DeleteTablet for 3 replicas of tablet a58afc299377437dac9a8670b297de4c
I20260808 07:11:44.581700 31297 tablet_service.cc:1514] Processing DeleteTablet for tablet a58afc299377437dac9a8670b297de4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:55572
I20260808 07:11:44.581784 31453 tablet_service.cc:1514] Processing DeleteTablet for tablet a58afc299377437dac9a8670b297de4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:54902
I20260808 07:11:44.581882 31584 tablet_replica.cc:330] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: stopping tablet replica
I20260808 07:11:44.581881 31586 tablet_replica.cc:330] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: stopping tablet replica
I20260808 07:11:44.581938 31584 raft_consensus.cc:2229] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:44.581949 31586 raft_consensus.cc:2229] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:44.581990 31584 raft_consensus.cc:2258] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.581998 31586 raft_consensus.cc:2258] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.582237 31142 tablet_service.cc:1514] Processing DeleteTablet for tablet a58afc299377437dac9a8670b297de4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:59792
I20260808 07:11:44.582322 31585 tablet_replica.cc:330] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: stopping tablet replica
I20260808 07:11:44.582278 30987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 07:11:44.582371 31585 raft_consensus.cc:2229] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:44.582387 31584 ts_tablet_manager.cc:1905] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:11:44.582417 30987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:44.582450 31585 raft_consensus.cc:2258] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.582751 31586 ts_tablet_manager.cc:1905] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:44.582844 31585 ts_tablet_manager.cc:1905] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:44.584324 31586 ts_tablet_manager.cc:1918] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:44.584337 31584 ts_tablet_manager.cc:1918] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:44.584399 31584 log.cc:1196] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/a58afc299377437dac9a8670b297de4c
I20260808 07:11:44.584385 31586 log.cc:1196] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/a58afc299377437dac9a8670b297de4c
I20260808 07:11:44.584404 31585 ts_tablet_manager.cc:1918] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:44.584451 31585 log.cc:1196] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/a58afc299377437dac9a8670b297de4c
I20260808 07:11:44.584646 31584 ts_tablet_manager.cc:1939] T a58afc299377437dac9a8670b297de4c P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting consensus metadata
I20260808 07:11:44.584645 31586 ts_tablet_manager.cc:1939] T a58afc299377437dac9a8670b297de4c P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting consensus metadata
I20260808 07:11:44.584659 31585 ts_tablet_manager.cc:1939] T a58afc299377437dac9a8670b297de4c P 9e212e004bc94353bb3f931696d09d06: Deleting consensus metadata
I20260808 07:11:44.585198 30970 catalog_manager.cc:4841] TS 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361): tablet a58afc299377437dac9a8670b297de4c (table test-table [id=ba0ebba7342949d7bc1ea2ab94ee1510]) successfully deleted
I20260808 07:11:44.585532 30972 catalog_manager.cc:4841] TS 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637): tablet a58afc299377437dac9a8670b297de4c (table test-table [id=ba0ebba7342949d7bc1ea2ab94ee1510]) successfully deleted
I20260808 07:11:44.586139 31297 tablet_service.cc:1467] Processing CreateTablet for tablet 213612937ba64c6db3fb56f940321f6d (DEFAULT_TABLE table=test-table [id=60443a483dae4f11b9b5550f87e292a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:44.586153 31453 tablet_service.cc:1467] Processing CreateTablet for tablet 213612937ba64c6db3fb56f940321f6d (DEFAULT_TABLE table=test-table [id=60443a483dae4f11b9b5550f87e292a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:44.586181 31142 tablet_service.cc:1467] Processing CreateTablet for tablet 213612937ba64c6db3fb56f940321f6d (DEFAULT_TABLE table=test-table [id=60443a483dae4f11b9b5550f87e292a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:44.586256 31453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213612937ba64c6db3fb56f940321f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.586256 31297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213612937ba64c6db3fb56f940321f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.586292 31142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213612937ba64c6db3fb56f940321f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:44.587471 31559 tablet_bootstrap.cc:492] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: Bootstrap starting.
I20260808 07:11:44.587466 31558 tablet_bootstrap.cc:492] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Bootstrap starting.
I20260808 07:11:44.587801 31559 tablet_bootstrap.cc:654] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.587893 31558 tablet_bootstrap.cc:654] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.588220 31562 tablet_bootstrap.cc:492] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06: Bootstrap starting.
I20260808 07:11:44.588428 31559 tablet_bootstrap.cc:492] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: No bootstrap required, opened a new log
I20260808 07:11:44.588485 31559 ts_tablet_manager.cc:1397] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:44.588649 31562 tablet_bootstrap.cc:654] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:44.588601 31559 raft_consensus.cc:348] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.588685 31559 raft_consensus.cc:374] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.588707 31559 raft_consensus.cc:729] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f95db4c74e04b11913c6f78fe8cbeff, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.588758 31559 consensus_queue.cc:260] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.588873 31559 ts_tablet_manager.cc:1428] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:44.589051 30969 catalog_manager.cc:4841] TS 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803): tablet a58afc299377437dac9a8670b297de4c (table test-table [id=ba0ebba7342949d7bc1ea2ab94ee1510]) successfully deleted
I20260808 07:11:44.589546 31562 tablet_bootstrap.cc:492] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06: No bootstrap required, opened a new log
I20260808 07:11:44.589588 31562 ts_tablet_manager.cc:1397] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:44.589694 31562 raft_consensus.cc:348] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.589741 31562 raft_consensus.cc:374] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.589756 31562 raft_consensus.cc:729] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e212e004bc94353bb3f931696d09d06, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.589818 31562 consensus_queue.cc:260] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.589921 31562 ts_tablet_manager.cc:1428] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:44.590320 31558 tablet_bootstrap.cc:492] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: No bootstrap required, opened a new log
I20260808 07:11:44.590374 31558 ts_tablet_manager.cc:1397] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:11:44.590512 31558 raft_consensus.cc:348] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.590593 31558 raft_consensus.cc:374] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:44.590621 31558 raft_consensus.cc:729] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a31cb83c8e4b0b8a8aaff5faedbc3c, State: Initialized, Role: FOLLOWER
I20260808 07:11:44.590669 31558 consensus_queue.cc:260] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.590766 31558 ts_tablet_manager.cc:1428] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:44.638170 31567 raft_consensus.cc:482] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:44.638281 31567 raft_consensus.cc:504] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.638479 31567 leader_election.cc:288] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361), 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637)
I20260808 07:11:44.638692 31321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213612937ba64c6db3fb56f940321f6d" candidate_uuid: "9e212e004bc94353bb3f931696d09d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" is_pre_election: true
I20260808 07:11:44.638724 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213612937ba64c6db3fb56f940321f6d" candidate_uuid: "9e212e004bc94353bb3f931696d09d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" is_pre_election: true
I20260808 07:11:44.638808 31321 raft_consensus.cc:2454] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e212e004bc94353bb3f931696d09d06 in term 0.
I20260808 07:11:44.638821 31474 raft_consensus.cc:2454] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e212e004bc94353bb3f931696d09d06 in term 0.
I20260808 07:11:44.638980 31088 leader_election.cc:302] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f95db4c74e04b11913c6f78fe8cbeff, 9e212e004bc94353bb3f931696d09d06; no voters:
I20260808 07:11:44.639101 31567 raft_consensus.cc:2781] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:44.639169 31567 raft_consensus.cc:482] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:44.639190 31567 raft_consensus.cc:3037] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.640074 31567 raft_consensus.cc:504] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.640215 31567 leader_election.cc:288] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [CANDIDATE]: Term 1 election: Requested vote from peers 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361), 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637)
I20260808 07:11:44.640401 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213612937ba64c6db3fb56f940321f6d" candidate_uuid: "9e212e004bc94353bb3f931696d09d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c"
I20260808 07:11:44.640489 31474 raft_consensus.cc:3037] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.641230 31321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213612937ba64c6db3fb56f940321f6d" candidate_uuid: "9e212e004bc94353bb3f931696d09d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f95db4c74e04b11913c6f78fe8cbeff"
I20260808 07:11:44.641315 31474 raft_consensus.cc:2454] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e212e004bc94353bb3f931696d09d06 in term 1.
I20260808 07:11:44.641335 31321 raft_consensus.cc:3037] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:44.641474 31090 leader_election.cc:302] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a31cb83c8e4b0b8a8aaff5faedbc3c, 9e212e004bc94353bb3f931696d09d06; no voters:
I20260808 07:11:44.641587 31567 raft_consensus.cc:2781] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:44.641654 31567 raft_consensus.cc:686] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 1 LEADER]: Becoming Leader. State: Replica: 9e212e004bc94353bb3f931696d09d06, State: Running, Role: LEADER
I20260808 07:11:44.641708 31567 consensus_queue.cc:237] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } }
I20260808 07:11:44.642230 31321 raft_consensus.cc:2454] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e212e004bc94353bb3f931696d09d06 in term 1.
I20260808 07:11:44.642284 30987 catalog_manager.cc:5495] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e212e004bc94353bb3f931696d09d06 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "9e212e004bc94353bb3f931696d09d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e212e004bc94353bb3f931696d09d06" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 44803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:44.653959 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.651251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:52069 (local address 127.22.2.130:44361)
0808 07:11:44.651293 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.651295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.651443 (+ 148us) server_negotiation.cc:407] Connection header received
0808 07:11:44.651520 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.651520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.651530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.651643 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:44.651823 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.652286 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.653207 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.653405 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.653479 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.653670 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.653726 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.653788 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.653856 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.653875 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:11:44.653999 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.651208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44361 (local address 127.22.2.129:52069)
0808 07:11:44.651350 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:44.651386 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:44.651459 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:44.651650 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:44.651651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:44.651652 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:44.651718 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.651720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.652374 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.652376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.653112 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.653114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.653477 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.653478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.653544 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.653567 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.653790 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.653793 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:44.653840 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:11:44.653885 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260808 07:11:44.656158 31321 raft_consensus.cc:1261] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Refusing update from remote peer 9e212e004bc94353bb3f931696d09d06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:44.656176 31474 raft_consensus.cc:1261] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Refusing update from remote peer 9e212e004bc94353bb3f931696d09d06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:44.656373 31567 consensus_queue.cc:1035] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f95db4c74e04b11913c6f78fe8cbeff" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:44.656508 31567 consensus_queue.cc:1035] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22a31cb83c8e4b0b8a8aaff5faedbc3c" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:44.663551 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.661200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:48907 (local address 127.22.2.129:44803)
0808 07:11:44.661237 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:44.661239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:44.661247 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:44.661269 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:44.661270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:44.661279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:44.661333 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:44.661493 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.662005 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.662739 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.663021 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.663087 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.663310 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.663345 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.663398 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:44.663465 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:11:44.663487 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:11:44.663550 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.661111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:44803 (local address 127.22.2.130:48907)
0808 07:11:44.661156 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:44.661168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:44.661206 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:44.661378 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:44.661379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:44.661380 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:44.661415 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.661416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.662088 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.662089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.662634 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:44.662636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:44.663140 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:44.663141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:44.663206 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:44.663213 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:44.663418 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:44.663420 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:44.663460 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 07:11:44.663500 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:11:44.672940 30987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:44.673056 30987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:44.673712 30987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 585cd0c117524972993736839cc8d517: Sending DeleteTablet for 3 replicas of tablet 213612937ba64c6db3fb56f940321f6d
I20260808 07:11:44.673920 31142 tablet_service.cc:1514] Processing DeleteTablet for tablet 213612937ba64c6db3fb56f940321f6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:59792
I20260808 07:11:44.673949 31297 tablet_service.cc:1514] Processing DeleteTablet for tablet 213612937ba64c6db3fb56f940321f6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:55572
I20260808 07:11:44.674014 31585 tablet_replica.cc:330] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06: stopping tablet replica
I20260808 07:11:44.674036 31586 tablet_replica.cc:330] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: stopping tablet replica
I20260808 07:11:44.674062 31585 raft_consensus.cc:2229] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:44.674088 31586 raft_consensus.cc:2229] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:44.674117 31585 raft_consensus.cc:2258] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.674117 31586 raft_consensus.cc:2258] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.674220 31453 tablet_service.cc:1514] Processing DeleteTablet for tablet 213612937ba64c6db3fb56f940321f6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:44 UTC) from {username='oryx', principal='oryx/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:54902
I20260808 07:11:44.674306 31584 tablet_replica.cc:330] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: stopping tablet replica
I20260808 07:11:44.674360 31584 raft_consensus.cc:2229] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:44.674399 31584 raft_consensus.cc:2258] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:44.674518 31585 ts_tablet_manager.cc:1905] T 213612937ba64c6db3fb56f940321f6d P 9e212e004bc94353bb3f931696d09d06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:44.674665 31586 ts_tablet_manager.cc:1905] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:44.675753 31584 ts_tablet_manager.cc:1905] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:44.676014 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 31071
I20260808 07:11:44.676208 31586 ts_tablet_manager.cc:1918] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:44.676254 31586 log.cc:1196] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/213612937ba64c6db3fb56f940321f6d
I20260808 07:11:44.676431 31586 ts_tablet_manager.cc:1939] T 213612937ba64c6db3fb56f940321f6d P 6f95db4c74e04b11913c6f78fe8cbeff: Deleting consensus metadata
I20260808 07:11:44.677522 31584 ts_tablet_manager.cc:1918] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:44.677575 31584 log.cc:1196] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/213612937ba64c6db3fb56f940321f6d
I20260808 07:11:44.677780 31584 ts_tablet_manager.cc:1939] T 213612937ba64c6db3fb56f940321f6d P 22a31cb83c8e4b0b8a8aaff5faedbc3c: Deleting consensus metadata
I20260808 07:11:44.677951 30970 catalog_manager.cc:4841] TS 6f95db4c74e04b11913c6f78fe8cbeff (127.22.2.130:44361): tablet 213612937ba64c6db3fb56f940321f6d (table test-table [id=60443a483dae4f11b9b5550f87e292a3]) successfully deleted
I20260808 07:11:44.678058 30972 catalog_manager.cc:4841] TS 22a31cb83c8e4b0b8a8aaff5faedbc3c (127.22.2.131:35637): tablet 213612937ba64c6db3fb56f940321f6d (table test-table [id=60443a483dae4f11b9b5550f87e292a3]) successfully deleted
W20260808 07:11:44.685294 30969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:44803 (error 108)
I20260808 07:11:44.685601 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 31226
I20260808 07:11:44.686846 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:44.685733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:44803 (local address 127.0.0.1:59810)
0808 07:11:44.686754 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:44.686801 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":999,"thread_start_us":62,"threads_started":1}
W20260808 07:11:44.686936 30969 catalog_manager.cc:4584] TS 9e212e004bc94353bb3f931696d09d06 (127.22.2.129:44803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 213612937ba64c6db3fb56f940321f6d: Network error: Client connection negotiation failed: client connection to 127.22.2.129:44803: connect: Connection refused (error 111)
I20260808 07:11:44.693821 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 31381
I20260808 07:11:44.700070 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 30948
2026-08-08T07:11:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3008 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:44 dist-test-slave-6k22 krb5kdc[31662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:44 dist-test-slave-6k22 krb5kdc[31662](info): set up 2 sockets
Aug 08 07:11:44 dist-test-slave-6k22 krb5kdc[31662](info): commencing operation
krb5kdc: starting...
W20260808 07:11:45.783077 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:45 dist-test-slave-6k22 krb5kdc[31662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:45.901903 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35711
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:41945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:45.983475 31677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:45.983755 31677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:45.983779 31677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:45.984881 31677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:45.984915 31677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:45.984929 31677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:45.984942 31677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:45.984954 31677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:45.986328 31677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:45.986511 31677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:45.986672 31677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:45.991437 31683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:45.991460 31682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:45.991437 31685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:45.991729 31677 server_base.cc:1034] running on GCE node
I20260808 07:11:45.991991 31677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:45.992285 31677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:45.993438 31677 hybrid_clock.cc:648] HybridClock initialized: now 1786173105993423 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T07:11:46Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1296 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:46 dist-test-slave-6k22 krb5kdc[31690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:46 dist-test-slave-6k22 krb5kdc[31690](info): set up 2 sockets
Aug 08 07:11:46 dist-test-slave-6k22 krb5kdc[31690](info): commencing operation
krb5kdc: starting...
W20260808 07:11:47.100397 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:11:47.222487 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40775
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:47.303278 31703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:47.303490 31703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:47.303512 31703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:47.304683 31703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:47.304726 31703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:47.304741 31703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:47.304754 31703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:47.304766 31703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:47.306131 31703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:47.306321 31703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:47.306484 31703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:47.311299 31712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:47.311307 31709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:47.311482 31703 server_base.cc:1034] running on GCE node
W20260808 07:11:47.311507 31710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:47.311830 31703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:47.312147 31703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:47.313331 31703 hybrid_clock.cc:648] HybridClock initialized: now 1786173107313330 us; error 55 us; skew 500 ppm
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:47.315847 31703 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:47.316315 31703 webserver.cc:460] Webserver started at http://127.22.2.190:40293/ using document root <none> and password file <none>
I20260808 07:11:47.316541 31703 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:47.316608 31703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:47.316738 31703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:47.317770 31703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "3a9c9495a4724d32909d48a58e285d22"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "901414f69172c88f3f2b6700b861a3f0"
server_key_iv: "1d4ad802c0a59230939b4c84547e4a9b"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.318153 31703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "3a9c9495a4724d32909d48a58e285d22"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "901414f69172c88f3f2b6700b861a3f0"
server_key_iv: "1d4ad802c0a59230939b4c84547e4a9b"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.319649 31703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:47.320446 31719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.320640 31703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:47.320747 31703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "3a9c9495a4724d32909d48a58e285d22"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "901414f69172c88f3f2b6700b861a3f0"
server_key_iv: "1d4ad802c0a59230939b4c84547e4a9b"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.320825 31703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:47.359287 31703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:47.361424 31703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:47.361601 31703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:47.365945 31703 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:40775
I20260808 07:11:47.365986 31776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:40775 every 8 connection(s)
I20260808 07:11:47.366389 31703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:47.367363 31777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:47.367730 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 31703
I20260808 07:11:47.367863 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:11:47.368079 22538 external_mini_cluster.cc:1428] Setting key ba3e3edcbb58e2a515014d2a924b89da
I20260808 07:11:47.370529 31777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Bootstrap starting.
I20260808 07:11:47.371176 31777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:47.371471 31777 log.cc:824] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:47.372181 31777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: No bootstrap required, opened a new log
I20260808 07:11:47.374449 31777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } }
I20260808 07:11:47.374632 31777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:47.374668 31777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9c9495a4724d32909d48a58e285d22, State: Initialized, Role: FOLLOWER
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:47.374830 31777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } }
I20260808 07:11:47.374909 31777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:47.374933 31777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:47.375007 31777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:47.375654 31777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } }
I20260808 07:11:47.375770 31777 leader_election.cc:302] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a9c9495a4724d32909d48a58e285d22; no voters:
I20260808 07:11:47.375972 31777 leader_election.cc:288] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:47.376004 31782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:47.376132 31782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 1 LEADER]: Becoming Leader. State: Replica: 3a9c9495a4724d32909d48a58e285d22, State: Running, Role: LEADER
I20260808 07:11:47.376323 31777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:47.376480 31782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } }
I20260808 07:11:47.376854 31782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a9c9495a4724d32909d48a58e285d22. Latest consensus state: current_term: 1 leader_uuid: "3a9c9495a4724d32909d48a58e285d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } } }
I20260808 07:11:47.376921 31782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:47.377000 31784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a9c9495a4724d32909d48a58e285d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } } }
I20260808 07:11:47.377107 31784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:47.377162 31790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:47.377705 31790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:47.379352 31790 catalog_manager.cc:1347] Generated new cluster ID: 06ac0d4b634b4de5b542fba5769ee980
I20260808 07:11:47.379405 31790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:47.379590 31780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:47.368641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.22.2.190:40775)
0808 07:11:47.368859 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:47.368867 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:47.368892 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:11:47.369285 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:47.369291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:47.369336 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:47.369452 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:47.370255 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.371236 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.372082 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.372573 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.372693 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:47.375327 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:47.375331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:47.375331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:47.375342 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:47.377686 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:47.378151 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:47.378154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:47.378154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:47.378262 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:47.378515 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:47.378516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:47.378516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:47.378844 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:47.379009 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:47.379132 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 07:11:47.379374 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
I20260808 07:11:47.385456 31790 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:47.386183 31790 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:47.389120 31790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Generated new TSK 0
I20260808 07:11:47.389289 31790 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:11:47.416453 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40775
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:47.500443 31805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:47.500650 31805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:47.500674 31805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:47.501865 31805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:47.501909 31805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:47.501935 31805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:47.503283 31805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:47.503458 31805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:47.503652 31805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:47.509104 31812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:47.509074 31815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:47.509132 31805 server_base.cc:1034] running on GCE node
W20260808 07:11:47.509186 31811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:47.509402 31805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:47.509595 31805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:47.510726 31805 hybrid_clock.cc:648] HybridClock initialized: now 1786173107510716 us; error 27 us; skew 500 ppm
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:47.513289 31805 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:11:47.513736 31805 webserver.cc:460] Webserver started at http://127.22.2.129:37815/ using document root <none> and password file <none>
I20260808 07:11:47.513909 31805 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:47.513948 31805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:47.514032 31805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:47.515056 31805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "9146c7cc724e4a1cb911ba80de1eb480"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "35e3d7f81d29eb0e8c9c3fe335e41db8"
server_key_iv: "0757438527a8b19aa81d70642ba2fdcb"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.515365 31805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "9146c7cc724e4a1cb911ba80de1eb480"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "35e3d7f81d29eb0e8c9c3fe335e41db8"
server_key_iv: "0757438527a8b19aa81d70642ba2fdcb"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.516706 31805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:47.517400 31823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.517580 31805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:47.517666 31805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "9146c7cc724e4a1cb911ba80de1eb480"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "35e3d7f81d29eb0e8c9c3fe335e41db8"
server_key_iv: "0757438527a8b19aa81d70642ba2fdcb"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.517719 31805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:47.541254 31805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:47.543622 31805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:47.543737 31805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:47.543933 31805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:47.544255 31805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:47.544288 31805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.544342 31805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:47.544368 31805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.550134 31805 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:42139
I20260808 07:11:47.550222 31945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:42139 every 8 connection(s)
I20260808 07:11:47.550535 31805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:47.551906 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 31805
I20260808 07:11:47.552001 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:47.552202 22538 external_mini_cluster.cc:1428] Setting key 1fc9fdd23703c124a6b615c91fce3792
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 07:11:47.562052 31780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:47.551300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:45027 (local address 127.22.2.190:40775)
0808 07:11:47.551433 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:47.551436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:47.551636 (+ 200us) server_negotiation.cc:407] Connection header received
0808 07:11:47.552307 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:47.552309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:47.552323 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:47.552402 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:47.553467 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.554216 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.555718 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.555927 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.555985 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:47.558754 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:47.558756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:47.558757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:47.558765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:47.560454 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:47.560949 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:47.560950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:47.560951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:47.561023 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:47.561400 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:47.561402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:47.561402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:47.561565 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:47.561651 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:47.561906 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 07:11:47.561980 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 07:11:47.562274 31948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:47.551247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:45027)
0808 07:11:47.551477 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:47.551525 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:47.552189 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:47.552485 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:47.552490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:47.553172 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:47.553371 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:47.553377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.554310 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.554311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:47.555609 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:47.555612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.556022 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.556024 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:47.556177 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:47.556809 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:47.556820 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:47.558638 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:47.560571 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:47.560580 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:47.560582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:47.560846 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:47.561111 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:47.561112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:47.561113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:47.561244 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:47.561643 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:47.561650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:47.561776 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 07:11:47.561870 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:11:47.562006 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
I20260808 07:11:47.562906 31946 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40775
I20260808 07:11:47.563006 31946 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:47.563179 31946 heartbeater.cc:511] Master 127.22.2.190:40775 requested a full tablet report, sending...
I20260808 07:11:47.563807 31737 ts_manager.cc:194] Registered new tserver with Master: 9146c7cc724e4a1cb911ba80de1eb480 (127.22.2.129:42139)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
I20260808 07:11:47.564640 31737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:45027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:11:47.578869 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40775
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:47.664633 31951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:47.664834 31951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:47.664857 31951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:47.666036 31951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:47.666080 31951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:47.666139 31951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:47.667719 31951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:47.667953 31951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:47.668185 31951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:47.673156 31957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:47.673218 31956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:47.673219 31960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:47.673537 31951 server_base.cc:1034] running on GCE node
I20260808 07:11:47.673725 31951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:47.673959 31951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:47.675099 31951 hybrid_clock.cc:648] HybridClock initialized: now 1786173107675085 us; error 52 us; skew 500 ppm
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:47.677688 31951 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:11:47.678052 31951 webserver.cc:460] Webserver started at http://127.22.2.130:43565/ using document root <none> and password file <none>
I20260808 07:11:47.678354 31951 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:47.678403 31951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:47.678493 31951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:47.679489 31951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "feb6d7aaed914642afd5ea5a2fc8bef7"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "4fbdb75f08045f19b459d55c59ee6d0d"
server_key_iv: "e40fb310b54254e16edf56d76bcf302a"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.679860 31951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "feb6d7aaed914642afd5ea5a2fc8bef7"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "4fbdb75f08045f19b459d55c59ee6d0d"
server_key_iv: "e40fb310b54254e16edf56d76bcf302a"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.681083 31951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:47.681789 31967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.681993 31951 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:47.682082 31951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "feb6d7aaed914642afd5ea5a2fc8bef7"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "4fbdb75f08045f19b459d55c59ee6d0d"
server_key_iv: "e40fb310b54254e16edf56d76bcf302a"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.682137 31951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:47.689044 31951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:47.691243 31951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:47.691401 31951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:47.691670 31951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:47.691989 31951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:47.692023 31951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.692073 31951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:47.692099 31951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.697929 31951 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41461
I20260808 07:11:47.698006 32096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41461 every 8 connection(s)
I20260808 07:11:47.698310 31951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:47.704973 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 31951
I20260808 07:11:47.705117 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:47.705349 22538 external_mini_cluster.cc:1428] Setting key 65979d75222e75339e73ff7673c44727
I20260808 07:11:47.707872 31780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:47.699034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:47769 (local address 127.22.2.190:40775)
0808 07:11:47.699078 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:47.699079 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:47.699331 (+ 252us) server_negotiation.cc:407] Connection header received
0808 07:11:47.699931 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:47.699932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:47.699945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:47.700008 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:47.700957 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.701455 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.702443 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.702641 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.702727 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:47.704775 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:47.704777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:47.704777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:47.704783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:47.706357 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:47.706763 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:47.706765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:47.706765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:47.706831 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:47.707082 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:47.707083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:47.707083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:47.707204 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:47.707283 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:47.707582 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 07:11:47.707667 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:11:47.708081 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:47.699019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:47769)
0808 07:11:47.699202 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:47.699243 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:47.699825 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:47.700083 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:47.700087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:47.700648 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:47.700844 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:47.700848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.701553 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.701555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:47.702349 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:47.702351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.702726 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.702727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:47.702850 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:47.703310 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:47.703319 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:47.704692 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:47.706439 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:47.706446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:47.706449 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:47.706675 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:47.706901 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:47.706902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:47.706903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:47.707011 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:47.707285 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:47.707293 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:47.707428 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 07:11:47.707535 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:11:47.707729 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260808 07:11:47.708571 32097 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40775
I20260808 07:11:47.708683 32097 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:47.708884 32097 heartbeater.cc:511] Master 127.22.2.190:40775 requested a full tablet report, sending...
I20260808 07:11:47.709277 31737 ts_manager.cc:194] Registered new tserver with Master: feb6d7aaed914642afd5ea5a2fc8bef7 (127.22.2.130:41461)
I20260808 07:11:47.709656 31737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:47769
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:11:47.732928 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40775
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:47.814692 32104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:47.814895 32104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:47.814918 32104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:47.816097 32104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:47.816138 32104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:47.816164 32104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:47.817608 32104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:47.817850 32104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:47.818071 32104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:47.823187 32111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:47.823262 32110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:47.823346 32104 server_base.cc:1034] running on GCE node
W20260808 07:11:47.823262 32113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:47.823668 32104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:47.823911 32104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:47.825047 32104 hybrid_clock.cc:648] HybridClock initialized: now 1786173107825037 us; error 27 us; skew 500 ppm
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:47.827464 32104 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:11:47.828029 32104 webserver.cc:460] Webserver started at http://127.22.2.131:32859/ using document root <none> and password file <none>
I20260808 07:11:47.828265 32104 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:47.828332 32104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:47.828485 32104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:47.829520 32104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "3d06e591b6204bd2892721734daefc80"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "06a2f9bad8bb625a66227ccaa2bb0ddc"
server_key_iv: "bf208142241bbcb0217255029ed599e7"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.829895 32104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "3d06e591b6204bd2892721734daefc80"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "06a2f9bad8bb625a66227ccaa2bb0ddc"
server_key_iv: "bf208142241bbcb0217255029ed599e7"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.831216 32104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:47.832008 32120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.832180 32104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.832279 32104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "3d06e591b6204bd2892721734daefc80"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "06a2f9bad8bb625a66227ccaa2bb0ddc"
server_key_iv: "bf208142241bbcb0217255029ed599e7"
server_key_version: "encryptionkey@0"
I20260808 07:11:47.832362 32104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:47.841324 32104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:47.843020 32104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:47.843178 32104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:47.843389 32104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:47.843788 32104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:47.843843 32104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.843876 32104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:47.843907 32104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:47.849682 32104 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:36273
I20260808 07:11:47.849757 32243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:36273 every 8 connection(s)
I20260808 07:11:47.850082 32104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:11:47 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:11:47.858168 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 32104
I20260808 07:11:47.858315 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:47.858522 22538 external_mini_cluster.cc:1428] Setting key 2c88d390f29148704c0856e0889127f6
I20260808 07:11:47.859428 31780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:47.850777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:51131 (local address 127.22.2.190:40775)
0808 07:11:47.850866 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:47.850868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:47.851065 (+ 197us) server_negotiation.cc:407] Connection header received
0808 07:11:47.851546 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:47.851547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:47.851615 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:47.851677 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:47.852591 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.853089 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.854012 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.854193 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.854252 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:47.856387 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:47.856388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:47.856388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:47.856394 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:47.857614 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:47.857967 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:47.857968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:47.857969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:47.858040 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:47.858475 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:47.858476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:47.858476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:47.858640 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:47.858708 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:47.859253 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 07:11:47.859337 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 07:11:47.859776 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:47.850740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:51131)
0808 07:11:47.850918 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:47.850952 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:47.851448 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:47.851766 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:47.851769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:47.852328 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:47.852511 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:47.852515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.853165 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.853165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:47.853933 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:47.853935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:47.854272 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:47.854273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:47.854376 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:47.854897 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:47.854905 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:47.856301 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:47.857686 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:47.857692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:47.857693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:47.857901 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:47.858196 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:47.858198 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:47.858198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:47.858330 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:47.858736 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:47.858743 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:47.859072 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 07:11:47.859214 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 07:11:47.859376 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
I20260808 07:11:47.860258 32244 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40775
I20260808 07:11:47.860364 32244 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:47.860555 32244 heartbeater.cc:511] Master 127.22.2.190:40775 requested a full tablet report, sending...
I20260808 07:11:47.860903 31737 ts_manager.cc:194] Registered new tserver with Master: 3d06e591b6204bd2892721734daefc80 (127.22.2.131:36273)
I20260808 07:11:47.861294 31737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:51131
I20260808 07:11:47.870396 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
Entry for principal oryx/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab.
WARNING: no policy specified for oryx/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
Entry for principal oryx/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab.
WARNING: no policy specified for oryx/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
Entry for principal oryx/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab.
I20260808 07:11:47.945734 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 31805
I20260808 07:11:47.953462 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 31951
I20260808 07:11:47.958936 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 32104
I20260808 07:11:47.964736 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 31703
I20260808 07:11:47.971817 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40775
--webserver_interface=127.22.2.190
--webserver_port=40293
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:48.054255 32255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:48.054467 32255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:48.054490 32255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:48.055649 32255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:48.055697 32255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:48.055712 32255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:48.055725 32255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:48.055784 32255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:48.057268 32255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=40293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:48.057535 32255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:48.057789 32255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:48.062624 32260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:48.062634 32261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:48.062634 32263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:48.062834 32255 server_base.cc:1034] running on GCE node
I20260808 07:11:48.063114 32255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:48.063367 32255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:48.064504 32255 hybrid_clock.cc:648] HybridClock initialized: now 1786173108064486 us; error 34 us; skew 500 ppm
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173108, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:48.067023 32255 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:11:48.067483 32255 webserver.cc:460] Webserver started at http://127.22.2.190:40293/ using document root <none> and password file <none>
I20260808 07:11:48.067752 32255 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:48.067842 32255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:48.069289 32255 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:48.069965 32271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.070147 32255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:48.070250 32255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "3a9c9495a4724d32909d48a58e285d22"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "901414f69172c88f3f2b6700b861a3f0"
server_key_iv: "1d4ad802c0a59230939b4c84547e4a9b"
server_key_version: "encryptionkey@0"
I20260808 07:11:48.070603 32255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:48.083461 32255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:48.085238 32255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:48.085402 32255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:48.089685 32255 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:40775
I20260808 07:11:48.089715 32335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:40775 every 8 connection(s)
I20260808 07:11:48.090095 32255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:48.091485 32336 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:11:48.092151 32336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Bootstrap starting.
I20260808 07:11:48.096896 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 32255
I20260808 07:11:48.097190 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:42139
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:40775
--webserver_port=37815
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.2.129
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:48.101364 32336 log.cc:824] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:48.103021 32336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:11:48.103339 32336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Bootstrap complete.
I20260808 07:11:48.105731 32336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } }
I20260808 07:11:48.105959 32336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9c9495a4724d32909d48a58e285d22, State: Initialized, Role: FOLLOWER
I20260808 07:11:48.106149 32336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } }
I20260808 07:11:48.106251 32336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:48.106307 32336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:48.106398 32336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:11:48.107465 32336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } }
I20260808 07:11:48.107599 32336 leader_election.cc:302] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a9c9495a4724d32909d48a58e285d22; no voters:
I20260808 07:11:48.107787 32336 leader_election.cc:288] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:11:48.107822 32340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:11:48.107941 32340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [term 2 LEADER]: Becoming Leader. State: Replica: 3a9c9495a4724d32909d48a58e285d22, State: Running, Role: LEADER
I20260808 07:11:48.108031 32340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } }
I20260808 07:11:48.108065 32336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:48.108285 32342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3a9c9495a4724d32909d48a58e285d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } } }
I20260808 07:11:48.108373 32342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:48.108613 32343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a9c9495a4724d32909d48a58e285d22. Latest consensus state: current_term: 2 leader_uuid: "3a9c9495a4724d32909d48a58e285d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9c9495a4724d32909d48a58e285d22" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40775 } } }
I20260808 07:11:48.108711 32343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:48.108771 32345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:48.109292 32345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:48.109483 32345 catalog_manager.cc:1259] Loaded cluster ID: 06ac0d4b634b4de5b542fba5769ee980
I20260808 07:11:48.109669 32345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:48.110597 32345 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:48.110797 32345 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3a9c9495a4724d32909d48a58e285d22: Loaded TSK: 0
I20260808 07:11:48.111012 32345 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:11:48.202657 32338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:48.202858 32338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:48.202883 32338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:48.204159 32338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:48.204202 32338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:48.204228 32338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:11:48.205631 32338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.129.keytab
--principal=oryx/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:42139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=37815
--tserver_master_addrs=127.22.2.190:40775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:48.205854 32338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:48.206100 32338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:48.212082 32366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:48.212082 32365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:48.212167 32368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:48.212487 32338 server_base.cc:1034] running on GCE node
I20260808 07:11:48.212692 32338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:48.212983 32338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:48.214139 32338 hybrid_clock.cc:648] HybridClock initialized: now 1786173108214119 us; error 37 us; skew 500 ppm
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173108, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:48.216707 32338 init.cc:377] Logged in from keytab as oryx/127.22.2.129@KRBTEST.COM (short username oryx)
I20260808 07:11:48.217193 32338 webserver.cc:460] Webserver started at http://127.22.2.129:37815/ using document root <none> and password file <none>
I20260808 07:11:48.217432 32338 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:48.217491 32338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:48.218897 32338 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:48.219597 32375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.219755 32338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:48.219848 32338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "9146c7cc724e4a1cb911ba80de1eb480"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "35e3d7f81d29eb0e8c9c3fe335e41db8"
server_key_iv: "0757438527a8b19aa81d70642ba2fdcb"
server_key_version: "encryptionkey@0"
I20260808 07:11:48.220189 32338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:48.248047 32338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:48.250428 32338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:48.250542 32338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:48.250871 32338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:48.251261 32338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:48.251299 32338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.251324 32338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:48.251338 32338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.258999 32338 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:42139
I20260808 07:11:48.259039 32501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:42139 every 8 connection(s)
I20260808 07:11:48.259413 32338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:48.263274 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 32338
I20260808 07:11:48.263649 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:41461
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:40775
--webserver_port=43565
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.2.130
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:48.268774 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.260109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:49141)
0808 07:11:48.260325 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.260375 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.260880 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.261330 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.261334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.261923 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.262112 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.262117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.263257 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.263258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.264267 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.264271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.265094 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.265095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.265252 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.265806 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.265817 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.268526 (+ 2709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":122,"threads_started":1}
W20260808 07:11:48.268867 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.260191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:49141 (local address 127.22.2.190:40775)
0808 07:11:48.260382 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.260390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.260465 (+ 75us) server_negotiation.cc:407] Connection header received
0808 07:11:48.261099 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.261104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.261146 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.261281 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.262236 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.263143 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.264351 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.264980 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.265098 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.268714 (+ 3616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:49141: BlockingRecv error: recv got EOF from 127.22.2.129:49141 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
W20260808 07:11:48.268985 32502 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:40775 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.2.190:40775: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:48.274000 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.269244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:36481)
0808 07:11:48.269294 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.269305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.269342 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.269476 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.269478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.269640 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.269699 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.269701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.270277 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.270279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.271161 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.271164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.271508 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.271510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.271604 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.271967 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.271973 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.273955 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:11:48.274401 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.269273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:36481 (local address 127.22.2.190:40775)
0808 07:11:48.269316 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.269318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.269328 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:48.269402 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.269402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.269411 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.269460 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.269772 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.270204 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.271260 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.271436 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.271484 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.274135 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:36481: BlockingRecv error: recv got EOF from 127.22.2.129:36481 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:48.278834 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.274280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:45441)
0808 07:11:48.274336 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.274383 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.274429 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.274707 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.274708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.274824 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.274848 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.274849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.275814 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.275814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.276269 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.276270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.276722 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.276723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.276757 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.277157 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.277161 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.278785 (+ 1624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 07:11:48.278975 32502 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:11:48.278996 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.274375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:45441 (local address 127.22.2.190:40775)
0808 07:11:48.274482 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.274484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.274493 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:48.274620 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.274620 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.274633 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.274699 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.274920 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.275729 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.276347 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.276648 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.276718 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.278881 (+ 2163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:45441: BlockingRecv error: recv got EOF from 127.22.2.129:45441 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":86,"threads_started":1}
W20260808 07:11:48.349371 32507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:48.349573 32507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:48.349597 32507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:48.350666 32507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:48.350701 32507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:48.350733 32507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:11:48.352097 32507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.130.keytab
--principal=oryx/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:41461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=43565
--tserver_master_addrs=127.22.2.190:40775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:48.352293 32507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:48.352460 32507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:48.357686 32513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:48.357693 32514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:48.357786 32507 server_base.cc:1034] running on GCE node
W20260808 07:11:48.357712 32516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:48.358199 32507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:48.358471 32507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:48.359624 32507 hybrid_clock.cc:648] HybridClock initialized: now 1786173108359604 us; error 49 us; skew 500 ppm
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173108, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:48.362114 32507 init.cc:377] Logged in from keytab as oryx/127.22.2.130@KRBTEST.COM (short username oryx)
I20260808 07:11:48.362623 32507 webserver.cc:460] Webserver started at http://127.22.2.130:43565/ using document root <none> and password file <none>
I20260808 07:11:48.362824 32507 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:48.362880 32507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:48.364275 32507 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:48.364850 32524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.365024 32507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:48.365126 32507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "feb6d7aaed914642afd5ea5a2fc8bef7"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "4fbdb75f08045f19b459d55c59ee6d0d"
server_key_iv: "e40fb310b54254e16edf56d76bcf302a"
server_key_version: "encryptionkey@0"
I20260808 07:11:48.365476 32507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:48.373095 32507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:48.374998 32507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:48.375165 32507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:48.375401 32507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:48.375787 32507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:48.375844 32507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.375890 32507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:48.375916 32507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.381865 32507 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:41461
I20260808 07:11:48.381937 32648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:41461 every 8 connection(s)
I20260808 07:11:48.382301 32507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:11:48.389544 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.383052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:48115 (local address 127.22.2.190:40775)
0808 07:11:48.383102 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.383104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.383367 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:11:48.383934 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.383935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.383950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.384003 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.384900 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.385384 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.386353 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.386532 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.386582 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.389487 (+ 2905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:48115: BlockingRecv error: recv got EOF from 127.22.2.130:48115 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:48.389600 32652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.383023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:48115)
0808 07:11:48.383183 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.383252 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.383824 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.384057 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.384060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.384628 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.384822 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.384827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.385465 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.385466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.386260 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.386263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.386621 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.386622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.386724 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.387184 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.387192 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.389403 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
W20260808 07:11:48.389822 32649 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:40775 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.2.190:40775: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
I20260808 07:11:48.390218 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 32507
I20260808 07:11:48.390556 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:36273
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:40775
--webserver_port=32859
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.2.131
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:48.394850 32652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.390093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:48033)
0808 07:11:48.390153 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.390163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.390199 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.390391 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.390391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.390502 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.390563 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.390564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.391416 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.391417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.391977 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.391979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.392451 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.392453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.392523 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.392921 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.392927 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.394765 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 07:11:48.394905 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.390149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:48033 (local address 127.22.2.190:40775)
0808 07:11:48.390212 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.390221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.390250 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:11:48.390276 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.390277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.390289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.390358 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.390660 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.391338 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.392053 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.392361 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.392429 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.394836 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:48033: BlockingRecv error: recv got EOF from 127.22.2.130:48033 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:48.399408 32652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.395221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:33979)
0808 07:11:48.395275 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.395286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.395333 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.395471 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.395472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.395659 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.395686 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.395688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.396318 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.396319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.396803 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.396804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.397159 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.397160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.397215 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.397662 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.397667 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.399325 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:11:48.399458 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.395313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:33979 (local address 127.22.2.190:40775)
0808 07:11:48.395348 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.395350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.395357 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:48.395382 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.395382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.395392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.395467 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.395786 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.396238 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.396889 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.397073 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.397158 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.399393 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:33979: BlockingRecv error: recv got EOF from 127.22.2.130:33979 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:11:48.399632 32649 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:11:48.476438 32653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:48.476642 32653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:48.476667 32653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:48.477847 32653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:48.477891 32653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:48.477918 32653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:11:48.479362 32653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/krb5kdc/oryx_127.22.2.131.keytab
--principal=oryx/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:36273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=32859
--tserver_master_addrs=127.22.2.190:40775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:48.479633 32653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:48.479880 32653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:48.485008 32660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:48.484995 32659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:48.485046 32653 server_base.cc:1034] running on GCE node
W20260808 07:11:48.484997 32662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:48.485466 32653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:48.485694 32653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:48.486845 32653 hybrid_clock.cc:648] HybridClock initialized: now 1786173108486825 us; error 36 us; skew 500 ppm
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173108, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:48.489406 32653 init.cc:377] Logged in from keytab as oryx/127.22.2.131@KRBTEST.COM (short username oryx)
I20260808 07:11:48.489835 32653 webserver.cc:460] Webserver started at http://127.22.2.131:32859/ using document root <none> and password file <none>
I20260808 07:11:48.490060 32653 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:48.490136 32653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:48.491483 32653 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:48.492117 32669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.492300 32653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.492400 32653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "3d06e591b6204bd2892721734daefc80"
format_stamp: "Formatted at 2026-08-08 07:11:47 on dist-test-slave-6k22"
server_key: "06a2f9bad8bb625a66227ccaa2bb0ddc"
server_key_iv: "bf208142241bbcb0217255029ed599e7"
server_key_version: "encryptionkey@0"
I20260808 07:11:48.492734 32653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:48.517668 32653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:48.519796 32653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:48.519964 32653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:48.520223 32653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:48.520546 32653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:48.520581 32653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.520623 32653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:48.520650 32653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:48.526614 32653 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:36273
I20260808 07:11:48.526683 325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:36273 every 8 connection(s)
I20260808 07:11:48.527015 32653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:11:48.534118 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.527779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:43069 (local address 127.22.2.190:40775)
0808 07:11:48.527849 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.527851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.528065 (+ 214us) server_negotiation.cc:407] Connection header received
0808 07:11:48.528552 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.528553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.528564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.528617 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.529555 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.530086 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.531065 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.531299 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.531367 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.534070 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:43069: BlockingRecv error: recv got EOF from 127.22.2.131:43069 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:11:48.534127 329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.527796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:43069)
0808 07:11:48.527952 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.527990 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.528463 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.528668 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.528671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.529235 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.529440 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.529445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.530181 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.530182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.530961 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.530963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.531386 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.531387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.531484 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.531953 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.531961 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.533985 (+ 2024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
W20260808 07:11:48.534359 327 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:40775 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.2.190:40775: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
I20260808 07:11:48.536986 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 32653
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:48.539145 329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.534611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:43189)
0808 07:11:48.534666 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.534676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.534718 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.534946 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.534947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.535074 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.535116 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.535117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.535811 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.535812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.536299 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.536301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.536678 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.536678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.536744 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.537189 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.537193 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.539082 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:11:48.539261 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.534786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:43189 (local address 127.22.2.190:40775)
0808 07:11:48.534843 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.534845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.534851 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:11:48.534868 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.534869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.534877 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.534940 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.535195 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.535728 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.536397 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.536595 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.536678 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.539162 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:43189: BlockingRecv error: recv got EOF from 127.22.2.131:43189 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:48 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:48.543908 329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.539517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:48105)
0808 07:11:48.539597 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:48.539607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:48.539654 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:48.539783 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:48.539784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:48.539910 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:48.539934 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.539935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.540542 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.540543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.541191 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:48.541193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.541581 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.541582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:48.541634 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.542023 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:48.542028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:48.543833 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 07:11:48.543955 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:48.539598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:48105 (local address 127.22.2.190:40775)
0808 07:11:48.539651 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.539652 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.539660 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:48.539713 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.539713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.539722 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.539778 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.540007 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.540459 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.541294 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.541494 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.541579 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.543896 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:48105: BlockingRecv error: recv got EOF from 127.22.2.131:48105 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 07:11:48.544137 327 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:11:48.545042 32508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:48.537427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.22.2.190:40775)
0808 07:11:48.537504 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:48.537505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:48.537667 (+ 162us) server_negotiation.cc:407] Connection header received
0808 07:11:48.537779 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:48.537780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:48.537791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:48.537851 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:48.538586 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.539242 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.540169 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:48.540383 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:48.540467 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:48.541595 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:48.541600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:48.541601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:48.541613 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:48.543596 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:48.543957 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:48.543961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:48.543961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:48.544207 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:48.544386 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:48.544387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:48.544388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:48.544637 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:48.544748 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:48.544828 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 07:11:48.544910 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 07:11:49 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:49 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:49.284123 331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:49.279321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:46717)
0808 07:11:49.279495 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:49.279507 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:49.279628 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:49.279858 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:49.279859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:49.280043 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:49.280097 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:49.280098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.280797 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.280798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:49.281379 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:49.281381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.281700 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.281701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:49.281743 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:49.282257 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:49.282261 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:49.284052 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
W20260808 07:11:49.284255 332 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:49.279403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:46717 (local address 127.22.2.190:40775)
0808 07:11:49.279526 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:49.279528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:49.279603 (+ 75us) server_negotiation.cc:407] Connection header received
0808 07:11:49.279753 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:49.279753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:49.279765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:49.279834 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:49.280181 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.280677 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.281452 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.281638 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.281688 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:49.284146 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:46717: BlockingRecv error: recv got EOF from 127.22.2.129:46717 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
Aug 08 07:11:49 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:49 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:49.404881 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:49.400041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:59437)
0808 07:11:49.400220 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:49.400233 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:49.400339 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:49.400502 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:49.400503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:49.400689 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:49.400737 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:49.400739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.401430 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.401431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:49.401980 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:49.401982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.402379 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.402380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:49.402440 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:49.402917 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:49.402921 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:49.404773 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
W20260808 07:11:49.404934 332 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:49.400140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:59437 (local address 127.22.2.190:40775)
0808 07:11:49.400220 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:49.400222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:49.400315 (+ 93us) server_negotiation.cc:407] Connection header received
0808 07:11:49.400406 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:49.400407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:49.400424 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:49.400494 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:49.400833 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.401333 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.402070 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.402269 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.402380 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:49.404867 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:59437: BlockingRecv error: recv got EOF from 127.22.2.130:59437 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 07:11:49 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:49 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:49.549417 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:49.544571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:40265)
0808 07:11:49.544722 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:49.544734 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:49.544797 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:49.544927 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:49.544927 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:49.545094 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:49.545146 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:49.545148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.545764 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.545765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:49.546319 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:49.546320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.546636 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.546637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:49.546675 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:49.547297 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:49.547303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:49.549344 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260808 07:11:49.549540 332 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:49.544705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:40265 (local address 127.22.2.190:40775)
0808 07:11:49.544783 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:49.544785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:49.544795 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:49.544847 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:49.544848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:49.544858 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:49.544908 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:49.545205 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.545695 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.546393 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:49.546574 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:49.546622 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:49.549440 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:40265: BlockingRecv error: recv got EOF from 127.22.2.131:40265 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 07:11:50 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:50 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:50.288906 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:50.284592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:58101)
0808 07:11:50.284752 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:50.284763 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:50.284816 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:50.284934 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:50.284935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:50.285095 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:50.285147 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:50.285149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.285790 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.285791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:50.286319 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:50.286321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.286659 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.286660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:50.286699 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:50.287164 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:50.287169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:50.288847 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
W20260808 07:11:50.289045 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:50.284676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:58101 (local address 127.22.2.190:40775)
0808 07:11:50.284810 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:50.284812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:50.284820 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:50.284862 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:50.284862 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:50.284872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:50.284931 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:50.285207 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.285706 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.286397 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.286593 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.286649 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:50.288951 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:58101: BlockingRecv error: recv got EOF from 127.22.2.129:58101 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
Aug 08 07:11:50 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:50 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:50.410101 339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:50.405411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:55399)
0808 07:11:50.405565 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:50.405576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:50.405642 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:50.405779 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:50.405780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:50.405948 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:50.406000 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:50.406002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.406613 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.406614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:50.407148 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:50.407149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.407456 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.407457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:50.407498 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:50.408088 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:50.408097 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:50.410042 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
W20260808 07:11:50.410198 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:50.405475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:55399 (local address 127.22.2.190:40775)
0808 07:11:50.405571 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:50.405573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:50.405625 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:11:50.405701 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:50.405701 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:50.405711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:50.405752 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:50.406064 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.406547 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.407216 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.407393 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.407442 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:50.410131 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:55399: BlockingRecv error: recv got EOF from 127.22.2.130:55399 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 07:11:50 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:50 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:50.554919 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:50.549901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:42863)
0808 07:11:50.550036 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:50.550055 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:50.550138 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:50.550342 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:50.550342 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:50.550509 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:50.550563 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:50.550564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.551222 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.551223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:50.551792 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:50.551794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.552173 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.552175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:50.552236 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:50.552737 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:50.552742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:50.554860 (+ 2118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
W20260808 07:11:50.555141 336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:50.550129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:42863 (local address 127.22.2.190:40775)
0808 07:11:50.550212 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:50.550214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:50.550222 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:50.550243 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:50.550244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:50.550254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:50.550325 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:50.550628 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.551128 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.551876 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:50.552054 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:50.552106 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:50.554991 (+ 2885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:42863: BlockingRecv error: recv got EOF from 127.22.2.131:42863 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 07:11:51 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:51 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:51.294654 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:51.289414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:44229)
0808 07:11:51.289576 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:51.289589 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:51.289689 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:51.289831 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:51.289832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:51.289991 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:51.290043 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:51.290044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.290797 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.290798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:51.291345 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:51.291346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.291747 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.291748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:51.291791 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:51.292315 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:51.292326 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:51.294539 (+ 2213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
W20260808 07:11:51.294848 343 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:51.289498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:44229 (local address 127.22.2.190:40775)
0808 07:11:51.289682 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:51.289684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:51.289693 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:51.289735 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:51.289736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:51.289746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:51.289803 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:51.290111 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.290717 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.291428 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.291680 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.291739 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:51.294658 (+ 2919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:44229: BlockingRecv error: recv got EOF from 127.22.2.129:44229 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
Aug 08 07:11:51 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:51 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:51.415752 344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:51.410635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:56957)
0808 07:11:51.410780 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:51.410791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:51.410859 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:51.410991 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:51.410991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:51.411161 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:51.411223 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:51.411225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.412005 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.412007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:51.412607 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:51.412609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.412949 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.412949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:51.412993 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:51.413602 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:51.413608 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:51.415681 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
W20260808 07:11:51.415879 343 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:51.410705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:56957 (local address 127.22.2.190:40775)
0808 07:11:51.410754 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:51.410756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:51.410843 (+ 87us) server_negotiation.cc:407] Connection header received
0808 07:11:51.410906 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:51.410906 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:51.410918 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:51.410981 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:51.411294 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.411891 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.412690 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.412878 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.412931 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:51.415780 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:56957: BlockingRecv error: recv got EOF from 127.22.2.130:56957 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 07:11:51 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:51 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:51.560788 345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:51.555457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:41163)
0808 07:11:51.555658 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:51.555670 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:51.555726 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:51.555882 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:51.555883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:51.556084 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:51.556147 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:51.556154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.557153 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.557154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:51.557786 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:51.557789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.558170 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.558171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:51.558212 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:51.558722 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:51.558732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:51.560721 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
W20260808 07:11:51.560981 343 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:51.555611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:41163 (local address 127.22.2.190:40775)
0808 07:11:51.555720 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:51.555722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:51.555735 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:11:51.555789 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:51.555790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:51.555819 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:51.555867 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:51.556248 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.557068 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.557884 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:51.558092 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:51.558165 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:51.560871 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:41163: BlockingRecv error: recv got EOF from 127.22.2.131:41163 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 07:11:52 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:52 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:52.306597 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:52.295297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:60471)
0808 07:11:52.295486 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:52.295498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:52.295600 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:52.295779 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:52.295780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:52.295965 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:52.296019 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:52.296020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.296838 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.296839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:52.297379 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:52.297381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.297829 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.297830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:52.297873 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:52.304407 (+ 6534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:52.304413 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:52.306497 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
W20260808 07:11:52.306774 347 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:52.295416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:60471 (local address 127.22.2.190:40775)
0808 07:11:52.295577 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:52.295579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:52.295598 (+ 19us) server_negotiation.cc:407] Connection header received
0808 07:11:52.295675 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:52.295675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:52.295687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:52.295764 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:52.296148 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.296754 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.297488 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.297728 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.297815 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:52.306599 (+ 8784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:60471: BlockingRecv error: recv got EOF from 127.22.2.129:60471 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
Aug 08 07:11:52 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:52 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:52.420677 348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:52.416206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:46819)
0808 07:11:52.416346 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:52.416359 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:52.416427 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:52.416553 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:52.416554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:52.416716 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:52.416764 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:52.416766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.417405 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.417406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:52.417923 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:52.417924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.418257 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.418258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:52.418297 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:52.418802 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:52.418807 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:52.420619 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
W20260808 07:11:52.420840 347 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:52.416272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:46819 (local address 127.22.2.190:40775)
0808 07:11:52.416359 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:52.416361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:52.416425 (+ 64us) server_negotiation.cc:407] Connection header received
0808 07:11:52.416470 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:52.416471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:52.416482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:52.416533 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:52.416830 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.417337 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.417993 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.418184 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.418259 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:52.420758 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:46819: BlockingRecv error: recv got EOF from 127.22.2.130:46819 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 07:11:52 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:52 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:52.565989 349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:52.561255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:48409)
0808 07:11:52.561443 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:52.561454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:52.561512 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:52.561648 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:52.561649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:52.561813 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:52.561871 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:52.561873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.562507 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.562508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:52.563107 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:52.563110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.563488 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.563489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:52.563552 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:52.564100 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:52.564106 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:52.565889 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":123,"threads_started":1}
W20260808 07:11:52.566044 347 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:52.561335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:48409 (local address 127.22.2.190:40775)
0808 07:11:52.561381 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:52.561384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:52.561513 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:11:52.561556 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:52.561556 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:52.561568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:52.561623 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:52.561942 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.562439 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.563203 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:52.563407 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:52.563489 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:52.565977 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:48409: BlockingRecv error: recv got EOF from 127.22.2.131:48409 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 07:11:53 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:53 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:53.312901 350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:53.307226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:45331)
0808 07:11:53.307420 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:53.307432 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:53.307495 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:53.307780 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:53.307781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:53.307953 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:53.308022 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:53.308024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.309131 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.309132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:53.309727 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:53.309729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.310112 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.310113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:53.310156 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:53.310615 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:53.310619 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:53.312785 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":122,"threads_started":1}
W20260808 07:11:53.312973 351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:53.307371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:45331 (local address 127.22.2.190:40775)
0808 07:11:53.307509 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:53.307511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:53.307523 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:11:53.307551 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:53.307552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:53.307679 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:53.307767 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:53.308165 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.309062 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.309833 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.310041 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.310101 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:53.312885 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:45331: BlockingRecv error: recv got EOF from 127.22.2.129:45331 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
Aug 08 07:11:53 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:53 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:53.425951 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:53.421134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:49215)
0808 07:11:53.421269 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:53.421281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:53.421351 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:53.421512 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:53.421513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:53.421678 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:53.421729 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:53.421730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.422583 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.422583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:53.423098 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:53.423100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.423534 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.423535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:53.423657 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:53.424148 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:53.424153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:53.425891 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
W20260808 07:11:53.426074 351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:53.421243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:49215 (local address 127.22.2.190:40775)
0808 07:11:53.421331 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:53.421333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:53.421345 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:11:53.421415 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:53.421416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:53.421427 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:53.421495 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:53.421837 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.422501 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.423192 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.423471 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.423530 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:53.425995 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:49215: BlockingRecv error: recv got EOF from 127.22.2.130:49215 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 07:11:53 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:53 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:53.571413 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:53.566532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:40251)
0808 07:11:53.566736 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:53.566748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:53.566817 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:53.566927 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:53.566928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:53.567102 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:53.567155 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:53.567157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.567796 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.567797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:53.568366 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:53.568368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.568729 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.568730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:53.568768 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:53.569316 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:53.569321 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:53.571353 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":72,"threads_started":1}
W20260808 07:11:53.571547 351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:53.566588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:40251 (local address 127.22.2.190:40775)
0808 07:11:53.566642 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:53.566645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:53.566827 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:11:53.566856 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:53.566856 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:53.566869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:53.566915 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:53.567211 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.567730 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.568466 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:53.568657 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:53.568714 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:53.571443 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:40251: BlockingRecv error: recv got EOF from 127.22.2.131:40251 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:11:54 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:54 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:54.318215 354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:54.313445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:52533)
0808 07:11:54.313620 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:54.313635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:54.313701 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:54.313860 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:54.313860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:54.314035 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:54.314084 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:54.314086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.314746 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.314747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:54.315264 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:54.315265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.315660 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.315661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:54.315707 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:54.316164 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:54.316169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:54.318149 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":76,"threads_started":1}
W20260808 07:11:54.318378 355 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:54.313516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:52533 (local address 127.22.2.190:40775)
0808 07:11:54.313684 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:54.313686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:54.313695 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:54.313763 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:54.313764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:54.313775 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:54.313837 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:54.314149 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.314654 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.315347 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.315541 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.315641 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:54.318258 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:52533: BlockingRecv error: recv got EOF from 127.22.2.129:52533 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
Aug 08 07:11:54 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:54 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:54.431326 356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:54.426439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:51495)
0808 07:11:54.426588 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:54.426602 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:54.426713 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:54.426907 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:54.426908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:54.427134 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:54.427179 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:54.427181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.427925 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.427926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:54.428471 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:54.428473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.428863 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.428864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:54.428923 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:54.429419 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:54.429425 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:54.431223 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
W20260808 07:11:54.431363 355 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:54.426479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:51495 (local address 127.22.2.190:40775)
0808 07:11:54.426527 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:54.426530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:54.426678 (+ 148us) server_negotiation.cc:407] Connection header received
0808 07:11:54.426816 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:54.426817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:54.426831 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:54.426896 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:54.427257 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.427845 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.428562 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.428770 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.428861 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:54.431301 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:51495: BlockingRecv error: recv got EOF from 127.22.2.130:51495 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 07:11:54 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:54 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:54.576606 357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:54.571997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:44949)
0808 07:11:54.572135 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:54.572154 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:54.572213 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:54.572348 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:54.572349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:54.572517 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:54.572566 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:54.572567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.573195 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.573196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:54.573719 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:54.573721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.574120 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.574121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:54.574190 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:54.574700 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:54.574704 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:54.576507 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
W20260808 07:11:54.576645 355 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:54.572061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:44949 (local address 127.22.2.190:40775)
0808 07:11:54.572125 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:54.572127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:54.572231 (+ 104us) server_negotiation.cc:407] Connection header received
0808 07:11:54.572256 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:54.572256 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:54.572269 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:54.572322 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:54.572626 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.573108 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.573849 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:54.574043 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:54.574119 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:54.576583 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:44949: BlockingRecv error: recv got EOF from 127.22.2.131:44949 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 07:11:55 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:55 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:55.323405 359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:55.318721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:48441)
0808 07:11:55.318933 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:55.318943 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:55.319033 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:55.319172 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:55.319173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:55.319344 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:55.319397 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:55.319400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.320116 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.320118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:55.320637 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:55.320639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.320968 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.320968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:55.321008 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:55.321446 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:55.321450 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:55.323345 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":128,"threads_started":1}
W20260808 07:11:55.323557 360 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:55.318811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48441 (local address 127.22.2.190:40775)
0808 07:11:55.318941 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:55.318943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:55.319028 (+ 85us) server_negotiation.cc:407] Connection header received
0808 07:11:55.319094 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:55.319095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:55.319107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:55.319162 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:55.319460 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.320027 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.320710 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.320903 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.320954 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:55.323457 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:48441: BlockingRecv error: recv got EOF from 127.22.2.129:48441 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
Aug 08 07:11:55 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:55 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:55.436523 361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:55.431852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:50529)
0808 07:11:55.432015 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:55.432027 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:55.432092 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:55.432243 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:55.432244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:55.432404 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:55.432456 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:55.432457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.433080 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.433081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:55.433610 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:55.433612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.433940 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.433941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:55.434000 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:55.434532 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:55.434537 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:55.436463 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
W20260808 07:11:55.436663 360 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:55.431939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:50529 (local address 127.22.2.190:40775)
0808 07:11:55.432006 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:55.432008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:55.432090 (+ 82us) server_negotiation.cc:407] Connection header received
0808 07:11:55.432140 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:55.432140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:55.432150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:55.432205 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:55.432520 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.433015 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.433684 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.433873 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.433926 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:55.436578 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:50529: BlockingRecv error: recv got EOF from 127.22.2.130:50529 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 07:11:55 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:55 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:55.581810 362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:55.577093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:50429)
0808 07:11:55.577239 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:55.577251 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:55.577314 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:55.577444 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:55.577444 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:55.577613 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:55.577668 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:55.577670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.578299 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.578300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:55.578860 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:55.578862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.579233 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.579234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:55.579288 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:55.579791 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:55.579795 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:55.581711 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
W20260808 07:11:55.581868 360 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:55.577163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:50429 (local address 127.22.2.190:40775)
0808 07:11:55.577211 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:55.577213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:55.577302 (+ 89us) server_negotiation.cc:407] Connection header received
0808 07:11:55.577359 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:55.577359 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:55.577372 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:55.577422 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:55.577734 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.578228 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.578940 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:55.579142 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:55.579196 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:55.581792 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:50429: BlockingRecv error: recv got EOF from 127.22.2.131:50429 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 07:11:56 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:56 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:56.328436 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:56.323912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:46497)
0808 07:11:56.324092 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:56.324102 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:56.324154 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:56.324299 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:56.324300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:56.324478 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:56.324530 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:56.324532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.325179 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.325179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:56.325710 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:56.325712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.326068 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.326069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:56.326113 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:56.326570 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:56.326574 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:56.328376 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
W20260808 07:11:56.328583 364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:56.324030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:46497 (local address 127.22.2.190:40775)
0808 07:11:56.324162 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:56.324164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:56.324173 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:56.324205 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:56.324206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:56.324222 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:56.324280 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:56.324600 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.325112 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.325796 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.325998 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.326055 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:56.328486 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:46497: BlockingRecv error: recv got EOF from 127.22.2.129:46497 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
Aug 08 07:11:56 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:56 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:56.441759 365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:56.437005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:60741)
0808 07:11:56.437143 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:56.437153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:56.437211 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:56.437346 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:56.437347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:56.437504 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:56.437554 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:56.437556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.438396 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.438397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:56.438924 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:56.438925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.439269 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.439270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:56.439321 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:56.439823 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:56.439827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:56.441700 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
W20260808 07:11:56.441922 364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:56.437058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:60741 (local address 127.22.2.190:40775)
0808 07:11:56.437153 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:56.437155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:56.437211 (+ 56us) server_negotiation.cc:407] Connection header received
0808 07:11:56.437259 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:56.437260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:56.437271 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:56.437332 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:56.437648 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.438308 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.438996 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.439179 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.439255 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:56.441841 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:60741: BlockingRecv error: recv got EOF from 127.22.2.130:60741 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 07:11:56 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:56 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:56.586779 366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:56.582327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:40239)
0808 07:11:56.582461 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:56.582477 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:56.582533 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:56.582662 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:56.582663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:56.582837 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:56.582888 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:56.582890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.583535 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.583536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:56.584083 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:56.584084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.584439 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.584439 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:56.584479 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:56.584898 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:56.584903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:56.586721 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
W20260808 07:11:56.586897 364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:56.582357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:40239 (local address 127.22.2.190:40775)
0808 07:11:56.582438 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:56.582440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:56.582534 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:11:56.582589 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:56.582590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:56.582604 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:56.582645 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:56.582951 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.583463 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.584161 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:56.584371 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:56.584428 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:56.586823 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:40239: BlockingRecv error: recv got EOF from 127.22.2.131:40239 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 07:11:57 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:57 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:57.340332 367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:57.329109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:56045)
0808 07:11:57.329282 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:57.329293 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:57.329375 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:57.329518 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:57.329519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:57.329693 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:57.329741 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:57.329743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.330669 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.330670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:57.331195 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:57.331196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.331533 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.331534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:57.331635 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:57.338369 (+ 6734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:57.338379 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:57.340260 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
W20260808 07:11:57.340477 368 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:57.329205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56045 (local address 127.22.2.190:40775)
0808 07:11:57.329317 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:57.329319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:57.329377 (+ 58us) server_negotiation.cc:407] Connection header received
0808 07:11:57.329434 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:57.329435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:57.329446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:57.329496 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:57.329807 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.330591 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.331275 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.331466 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.331521 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:57.340384 (+ 8863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:56045: BlockingRecv error: recv got EOF from 127.22.2.129:56045 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":52,"threads_started":1}
Aug 08 07:11:57 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:57 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:57.446638 369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:57.442183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:60813)
0808 07:11:57.442324 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:57.442335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:57.442392 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:57.442515 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:57.442515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:57.442686 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:57.442735 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:57.442736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.443369 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.443370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:57.443941 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:57.443944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.444287 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.444288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:57.444336 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:57.444827 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:57.444832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:57.446568 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
W20260808 07:11:57.446777 368 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:57.442273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:60813 (local address 127.22.2.190:40775)
0808 07:11:57.442326 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:57.442328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:57.442379 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:11:57.442446 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:57.442447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:57.442456 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:57.442499 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:57.442802 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.443301 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.444016 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.444206 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.444277 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:57.446692 (+ 2415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:60813: BlockingRecv error: recv got EOF from 127.22.2.130:60813 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 07:11:57 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:57 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:57.592521 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:57.587309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:52571)
0808 07:11:57.587534 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:57.587546 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:57.587666 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:57.587863 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:57.587864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:57.588064 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:57.588112 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:57.588114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.588921 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.588922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:57.589652 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:57.589655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.590112 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.590113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:57.590158 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:57.590578 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:57.590582 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:57.592455 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
W20260808 07:11:57.592657 368 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:57.587412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:52571 (local address 127.22.2.190:40775)
0808 07:11:57.587516 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:57.587518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:57.587644 (+ 126us) server_negotiation.cc:407] Connection header received
0808 07:11:57.587734 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:57.587735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:57.587751 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:57.587849 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:57.588197 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.588837 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.589751 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:57.590033 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:57.590100 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:57.592549 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:52571: BlockingRecv error: recv got EOF from 127.22.2.131:52571 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 07:11:58 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:58 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:58.345872 371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:58.340850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:34743)
0808 07:11:58.341013 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:58.341023 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:58.341074 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:58.341231 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:58.341232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:58.341402 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:58.341452 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:58.341454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.342105 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.342106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:58.342634 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:58.342636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.342997 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.342999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:58.343051 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:58.343747 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:58.343754 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:58.345789 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
W20260808 07:11:58.345966 372 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:58.340945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34743 (local address 127.22.2.190:40775)
0808 07:11:58.341089 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:58.341091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:58.341100 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:58.341124 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:58.341124 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:58.341135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:58.341198 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:58.341524 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.342035 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.342707 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.342903 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.342956 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:58.345894 (+ 2938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:34743: BlockingRecv error: recv got EOF from 127.22.2.129:34743 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
Aug 08 07:11:58 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:58 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:58.452343 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:58.447125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:53863)
0808 07:11:58.447337 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:58.447350 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:58.447422 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:58.447706 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:58.447707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:58.447889 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:58.447956 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:58.447958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.448672 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.448673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:58.449281 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:58.449282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.449658 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.449659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:58.449708 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:58.450183 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:58.450188 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:58.452277 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
W20260808 07:11:58.452575 372 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:58.447337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53863 (local address 127.22.2.190:40775)
0808 07:11:58.447407 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:58.447408 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:58.447417 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:58.447484 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:58.447484 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:58.447495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:58.447544 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:58.448033 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.448529 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.449360 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.449540 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.449591 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:58.452507 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:53863: BlockingRecv error: recv got EOF from 127.22.2.130:53863 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 07:11:58 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:58 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:58.597906 374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:58.592966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:43769)
0808 07:11:58.593191 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:58.593204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:58.593290 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:58.593440 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:58.593441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:58.593627 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:58.593680 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:58.593682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.594372 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.594373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:58.594896 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:58.594897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.595325 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.595326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:58.595404 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:58.595945 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:58.595950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:58.597808 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
W20260808 07:11:58.597966 372 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:58.593078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:43769 (local address 127.22.2.190:40775)
0808 07:11:58.593194 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:58.593196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:58.593268 (+ 72us) server_negotiation.cc:407] Connection header received
0808 07:11:58.593366 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:58.593367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:58.593382 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:58.593436 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:58.593760 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.594279 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.594980 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:58.595237 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:58.595320 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:58.597901 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:43769: BlockingRecv error: recv got EOF from 127.22.2.131:43769 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 07:11:59 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:59 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:59.351687 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:59.346494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:58255)
0808 07:11:59.346658 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:59.346678 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:59.346738 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:59.346881 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:59.346882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:59.347073 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:59.347131 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:59.347133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.347812 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.347813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:59.348416 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:59.348418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.348780 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.348781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:59.348829 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:59.349405 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:59.349410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:59.351602 (+ 2192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
W20260808 07:11:59.351835 376 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:59.346556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:58255 (local address 127.22.2.190:40775)
0808 07:11:59.346697 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:59.346699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:59.346727 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:11:59.346798 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:59.346798 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:59.346810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:59.346868 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:59.347193 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.347733 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.348501 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.348697 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.348770 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:59.351706 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:58255: BlockingRecv error: recv got EOF from 127.22.2.129:58255 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
Aug 08 07:11:59 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:59 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:59.457315 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:59.452891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:34611)
0808 07:11:59.453079 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:59.453090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:59.453156 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:59.453296 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:59.453297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:59.453464 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:59.453514 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:59.453515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.454136 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.454137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:59.454655 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:59.454656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.454978 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.454979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:59.455021 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:59.455456 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:59.455461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:59.457236 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":117,"threads_started":1}
W20260808 07:11:59.457422 376 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:59.452988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:34611 (local address 127.22.2.190:40775)
0808 07:11:59.453054 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:59.453056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:59.453141 (+ 85us) server_negotiation.cc:407] Connection header received
0808 07:11:59.453215 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:59.453216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:59.453228 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:59.453270 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:59.453580 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.454069 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.454726 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.454914 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.454965 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:59.457326 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:34611: BlockingRecv error: recv got EOF from 127.22.2.130:34611 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 07:11:59 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:59 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:59.603494 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:59.598472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:40989)
0808 07:11:59.598621 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:59.598632 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:59.598693 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:59.598888 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:59.598889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:59.599071 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:59.599123 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:59.599124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.599850 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.599851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:59.600591 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:59.600594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.601034 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.601035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:59.601081 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:59.601559 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:59.601564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:59.603420 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260808 07:11:59.603611 376 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:59.598568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:40989 (local address 127.22.2.190:40775)
0808 07:11:59.598637 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:59.598639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:59.598691 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:11:59.598785 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:59.598786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:59.598798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:59.598865 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:59.599200 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.599767 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.600674 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:59.600966 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:59.601022 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:59.603513 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:40989: BlockingRecv error: recv got EOF from 127.22.2.131:40989 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 07:12:00 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:00 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:00.356993 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:00.352181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:50087)
0808 07:12:00.352352 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:00.352364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:00.352429 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:00.352577 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:00.352578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:00.352749 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:00.352797 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:00.352799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.353491 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.353492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:00.354020 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:00.354022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.354410 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.354411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:00.354454 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:00.354988 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:00.354993 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:00.356916 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":111,"threads_started":1}
W20260808 07:12:00.357193 380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:00.352238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:50087 (local address 127.22.2.190:40775)
0808 07:12:00.352395 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:00.352397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:00.352416 (+ 19us) server_negotiation.cc:407] Connection header received
0808 07:12:00.352492 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:00.352492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:00.352504 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:00.352573 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:00.352876 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.353412 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.354096 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.354338 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.354393 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:00.357054 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:50087: BlockingRecv error: recv got EOF from 127.22.2.129:50087 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
Aug 08 07:12:00 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:00 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:00.462565 381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:00.457766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:32827)
0808 07:12:00.457952 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:00.457964 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:00.458035 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:00.458166 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:00.458167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:00.458377 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:00.458424 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:00.458426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.459106 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.459107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:00.459720 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:00.459722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.460097 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.460097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:00.460157 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:00.460607 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:00.460611 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:00.462453 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":111,"threads_started":1}
W20260808 07:12:00.462618 380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:00.457861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:32827 (local address 127.22.2.190:40775)
0808 07:12:00.457936 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:00.457938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:00.458030 (+ 92us) server_negotiation.cc:407] Connection header received
0808 07:12:00.458095 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:00.458096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:00.458109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:00.458160 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:00.458509 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.459021 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.459803 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.460013 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.460096 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:00.462548 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:32827: BlockingRecv error: recv got EOF from 127.22.2.130:32827 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 07:12:00 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:00 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:00.609431 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:00.604076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:51287)
0808 07:12:00.604313 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:00.604326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:00.604413 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:00.604589 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:00.604590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:00.604792 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:00.604841 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:00.604843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.605542 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.605543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:00.606094 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:00.606096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.606494 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.606495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:00.606557 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:00.607087 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:00.607094 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:00.609317 (+ 2223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":110,"threads_started":1}
W20260808 07:12:00.609474 380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:00.604203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:51287 (local address 127.22.2.190:40775)
0808 07:12:00.604304 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:00.604306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:00.604407 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:12:00.604481 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:00.604482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:00.604503 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:00.604579 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:00.604928 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.605443 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.606216 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:00.606414 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:00.606495 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:00.609406 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:51287: BlockingRecv error: recv got EOF from 127.22.2.131:51287 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 07:12:01 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:01 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:01.362504 386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:01.357700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:43243)
0808 07:12:01.357868 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:01.357880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:01.357944 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:01.358097 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:01.358098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:01.358303 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:01.358375 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:01.358377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.359090 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.359091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:01.359684 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:01.359687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.360073 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.360073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:01.360113 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:01.360597 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:01.360601 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:01.362444 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
W20260808 07:12:01.362638 385 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:01.357681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:43243 (local address 127.22.2.190:40775)
0808 07:12:01.357846 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:01.357849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:01.357947 (+ 98us) server_negotiation.cc:407] Connection header received
0808 07:12:01.357988 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:01.357988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:01.358006 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:01.358080 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:01.358486 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.359016 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.359793 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.359997 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.360066 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:01.362547 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:43243: BlockingRecv error: recv got EOF from 127.22.2.129:43243 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
Aug 08 07:12:01 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:01 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:01.467674 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:01.463045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:46361)
0808 07:12:01.463183 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:01.463194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:01.463243 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:01.463374 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:01.463375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:01.463546 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:01.463614 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:01.463616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.464248 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.464249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:01.464777 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:01.464778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.465095 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.465095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:01.465136 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:01.465581 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:01.465586 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:01.467497 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
W20260808 07:12:01.467706 385 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:01.463164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:46361 (local address 127.22.2.190:40775)
0808 07:12:01.463219 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:01.463220 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:01.463229 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:12:01.463302 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:01.463302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:01.463313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:01.463367 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:01.463684 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.464180 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.464847 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.465030 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.465082 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:01.467659 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:46361: BlockingRecv error: recv got EOF from 127.22.2.130:46361 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 07:12:01 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:01 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:01.614962 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:01.609921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:56919)
0808 07:12:01.610065 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:01.610080 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:01.610154 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:01.610349 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:01.610350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:01.610539 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:01.610587 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:01.610589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.611294 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.611296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:01.611886 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:01.611888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.612256 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.612257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:01.612326 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:01.612872 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:01.612878 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:01.614854 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
W20260808 07:12:01.615005 385 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:01.610033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:56919 (local address 127.22.2.190:40775)
0808 07:12:01.610106 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:01.610108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:01.610133 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:12:01.610234 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:01.610235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:01.610249 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:01.610321 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:01.610670 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.611198 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.611975 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:01.612171 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:01.612245 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:01.614937 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:56919: BlockingRecv error: recv got EOF from 127.22.2.131:56919 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 07:12:02 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:02 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:02.372653 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:02.363118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:56725)
0808 07:12:02.363298 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:02.363310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:02.363370 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:02.363515 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:02.363515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:02.363709 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:02.363771 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:02.363772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.364438 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.364439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:02.365000 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:02.365001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.365330 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.365331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:02.365372 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:02.370565 (+ 5193us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:02.370571 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:02.372578 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
W20260808 07:12:02.372838 391 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:02.363231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56725 (local address 127.22.2.190:40775)
0808 07:12:02.363378 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:02.363381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:02.363390 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:12:02.363413 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:02.363414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:02.363425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:02.363486 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:02.363858 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.364365 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.365075 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.365263 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.365321 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:02.372727 (+ 7406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:56725: BlockingRecv error: recv got EOF from 127.22.2.129:56725 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
Aug 08 07:12:02 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:02 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:02.472909 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:02.468200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:36127)
0808 07:12:02.468348 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:02.468358 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:02.468424 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:02.468552 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:02.468553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:02.468726 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:02.468774 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:02.468776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.469504 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.469505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:02.470076 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:02.470078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.470425 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.470426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:02.470465 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:02.470929 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:02.470934 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:02.472837 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
W20260808 07:12:02.473006 391 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:02.468248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:36127 (local address 127.22.2.190:40775)
0808 07:12:02.468312 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:02.468314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:02.468421 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:12:02.468469 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:02.468470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:02.468482 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:02.468534 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:02.468840 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.469420 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.470168 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.470359 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.470415 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:02.472937 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:36127: BlockingRecv error: recv got EOF from 127.22.2.130:36127 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 07:12:02 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:02 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.131@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:02.620577 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:02.615599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.131:34499)
0808 07:12:02.615785 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:02.615798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:02.615915 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:02.616087 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:02.616088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:02.616336 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:02.616401 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:02.616403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.617132 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.617134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:02.617902 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:02.617904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.618225 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.618226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:02.618265 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:02.618723 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:02.618728 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:02.620520 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
W20260808 07:12:02.620751 391 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:02.615630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:34499 (local address 127.22.2.190:40775)
0808 07:12:02.615735 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:02.615737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:02.615892 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:12:02.615981 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:02.615981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:02.616002 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:02.616077 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:02.616478 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.617016 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.617967 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:02.618154 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:02.618213 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:02.620627 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.131:34499: BlockingRecv error: recv got EOF from 127.22.2.131:34499 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 07:12:03 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:03 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.129@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:03.378394 395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:03.373385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.129:32845)
0808 07:12:03.373527 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:03.373539 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:03.373600 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:03.373739 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:03.373740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:03.373911 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:03.373962 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:03.373970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.374705 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.374707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:03.375286 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:03.375288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.375649 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.375650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:03.375700 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:03.376247 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:03.376252 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:03.378269 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
W20260808 07:12:03.378449 396 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:03.373479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:32845 (local address 127.22.2.190:40775)
0808 07:12:03.373605 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:03.373607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:03.373619 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:12:03.373653 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:03.373654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:03.373665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:03.373724 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:03.374039 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.374606 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.375361 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.375578 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.375630 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:03.378375 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.129:32845: BlockingRecv error: recv got EOF from 127.22.2.129:32845 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
Aug 08 07:12:03 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:12:03 dist-test-slave-6k22 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.2.130@KRBTEST.COM for oryx/127.22.2.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:12:03.478955 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:03.473566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40775 (local address 127.22.2.130:58777)
0808 07:12:03.473718 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:03.473732 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:03.473789 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:03.474004 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:03.474006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:03.474305 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:03.474363 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:03.474365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.475202 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.475204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:03.475847 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:03.475849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.476282 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.476283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:03.476354 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:03.476878 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:03.476883 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:03.478854 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.2.190:40775: Server oryx/127.22.2.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
W20260808 07:12:03.479007 396 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:03.473611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:58777 (local address 127.22.2.190:40775)
0808 07:12:03.473754 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:03.473756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:03.473780 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:12:03.473882 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:03.473883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:03.473899 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:03.473993 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:03.474457 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.475100 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.475950 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.476181 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.476281 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:03.478939 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.2.130:58777: BlockingRecv error: recv got EOF from 127.22.2.130:58777 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:12:03.538726 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 32338
I20260808 07:12:03.545710 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 32507
I20260808 07:12:03.552315 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 32653
I20260808 07:12:03.557986 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 32255
2026-08-08T07:12:03Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17560 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T07:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:12:03Z Disabled control of system clock
I20260808 07:12:03.588824 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34651
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:34651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:12:03.669088 402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:03.669287 402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:03.669307 402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:03.670394 402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:12:03.670429 402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:03.670442 402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:03.670454 402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:12:03.670466 402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:12:03.671836 402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:34651
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:34651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:03.672037 402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:03.672420 402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:12:03.677781 402 server_base.cc:1034] running on GCE node
W20260808 07:12:03.677733 410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:03.677701 408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:03.677698 407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:03.678320 402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:03.678617 402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:03.679751 402 hybrid_clock.cc:648] HybridClock initialized: now 1786173123679739 us; error 36 us; skew 500 ppm
I20260808 07:12:03.680907 402 webserver.cc:460] Webserver started at http://127.22.2.190:36833/ using document root <none> and password file <none>
I20260808 07:12:03.681150 402 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:03.681196 402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:03.681375 402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:03.682422 402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "a54509f89e90482ab52aa3d0c53c81c8"
format_stamp: "Formatted at 2026-08-08 07:12:03 on dist-test-slave-6k22"
server_key: "d6e4f534ada89004cf7777c7bb621aab"
server_key_iv: "9674f3f5e867c9aede942db29124daba"
server_key_version: "encryptionkey@0"
I20260808 07:12:03.682796 402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "a54509f89e90482ab52aa3d0c53c81c8"
format_stamp: "Formatted at 2026-08-08 07:12:03 on dist-test-slave-6k22"
server_key: "d6e4f534ada89004cf7777c7bb621aab"
server_key_iv: "9674f3f5e867c9aede942db29124daba"
server_key_version: "encryptionkey@0"
I20260808 07:12:03.684130 402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:12:03.684834 417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:03.684999 402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:12:03.685083 402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "a54509f89e90482ab52aa3d0c53c81c8"
format_stamp: "Formatted at 2026-08-08 07:12:03 on dist-test-slave-6k22"
server_key: "d6e4f534ada89004cf7777c7bb621aab"
server_key_iv: "9674f3f5e867c9aede942db29124daba"
server_key_version: "encryptionkey@0"
I20260808 07:12:03.685149 402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:03.708091 402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:03.708410 402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:03.708559 402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:03.712491 402 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:34651
I20260808 07:12:03.712649 473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:34651 every 8 connection(s)
I20260808 07:12:03.712884 402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:12:03.713202 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 402
I20260808 07:12:03.713317 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:12:03.713547 22538 external_mini_cluster.cc:1428] Setting key fccedf1e8782ba2ee55d5ded91483081
I20260808 07:12:03.713785 474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:03.717531 474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8: Bootstrap starting.
I20260808 07:12:03.718254 474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:03.718616 474 log.cc:824] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:03.719452 474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8: No bootstrap required, opened a new log
I20260808 07:12:03.719763 477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:03.714262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.22.2.190:34651)
0808 07:12:03.714465 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:03.714475 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:03.714509 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:12:03.715037 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:03.715046 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:03.715129 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:03.715281 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:03.715475 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.716835 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.717550 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.718163 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.718288 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:03.719007 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:03.719011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:03.719011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:12:03.719215 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:03.719260 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:12:03.719276 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:03.719345 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:03.719414 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:12:03.719491 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260808 07:12:03.721359 474 raft_consensus.cc:348] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54509f89e90482ab52aa3d0c53c81c8" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34651 } }
I20260808 07:12:03.721493 474 raft_consensus.cc:374] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:03.721515 474 raft_consensus.cc:729] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a54509f89e90482ab52aa3d0c53c81c8, State: Initialized, Role: FOLLOWER
I20260808 07:12:03.721659 474 consensus_queue.cc:260] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54509f89e90482ab52aa3d0c53c81c8" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34651 } }
I20260808 07:12:03.721745 474 raft_consensus.cc:388] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:12:03.721796 474 raft_consensus.cc:482] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:12:03.721844 474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:03.722378 474 raft_consensus.cc:504] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54509f89e90482ab52aa3d0c53c81c8" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34651 } }
I20260808 07:12:03.722488 474 leader_election.cc:302] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a54509f89e90482ab52aa3d0c53c81c8; no voters:
I20260808 07:12:03.722643 474 leader_election.cc:288] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:12:03.722703 479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:12:03.722865 479 raft_consensus.cc:686] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [term 1 LEADER]: Becoming Leader. State: Replica: a54509f89e90482ab52aa3d0c53c81c8, State: Running, Role: LEADER
I20260808 07:12:03.722936 474 sys_catalog.cc:564] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:12:03.722951 479 consensus_queue.cc:237] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54509f89e90482ab52aa3d0c53c81c8" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34651 } }
I20260808 07:12:03.723387 481 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a54509f89e90482ab52aa3d0c53c81c8. Latest consensus state: current_term: 1 leader_uuid: "a54509f89e90482ab52aa3d0c53c81c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54509f89e90482ab52aa3d0c53c81c8" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34651 } } }
I20260808 07:12:03.723498 481 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:12:03.723364 480 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a54509f89e90482ab52aa3d0c53c81c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54509f89e90482ab52aa3d0c53c81c8" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 34651 } } }
I20260808 07:12:03.723634 480 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:12:03.724552 495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:12:03.725143 495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:12:03.726488 495 catalog_manager.cc:1347] Generated new cluster ID: f070786b586f4378b0a2864068b4bc62
I20260808 07:12:03.726541 495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:12:03.735833 495 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:12:03.736348 495 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:12:03.742851 495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a54509f89e90482ab52aa3d0c53c81c8: Generated new TSK 0
I20260808 07:12:03.743021 495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:12:03.747321 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34651
--builtin_ntp_servers=127.22.2.148:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 07:12:03.842721 501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:03.842922 501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:03.842943 501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:03.844129 501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:03.844169 501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:03.844192 501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:12:03.845512 501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:03.845777 501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:03.846071 501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:12:03.852005 501 server_base.cc:1034] running on GCE node
W20260808 07:12:03.852085 509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:03.851963 507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:03.851941 506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:03.852480 501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:03.852766 501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:03.853969 501 hybrid_clock.cc:648] HybridClock initialized: now 1786173123853963 us; error 34 us; skew 500 ppm
I20260808 07:12:03.855345 501 webserver.cc:460] Webserver started at http://127.22.2.129:42963/ using document root <none> and password file <none>
I20260808 07:12:03.855616 501 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:03.855674 501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:03.855795 501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:03.857125 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "24d2798fe1cc426492745b79c2ed7ed1"
format_stamp: "Formatted at 2026-08-08 07:12:03 on dist-test-slave-6k22"
server_key: "d59c20a9add46694e8bb593e68000283"
server_key_iv: "8d1ee37a21e46135fc22f9965ca9b2e3"
server_key_version: "encryptionkey@0"
I20260808 07:12:03.857502 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "24d2798fe1cc426492745b79c2ed7ed1"
format_stamp: "Formatted at 2026-08-08 07:12:03 on dist-test-slave-6k22"
server_key: "d59c20a9add46694e8bb593e68000283"
server_key_iv: "8d1ee37a21e46135fc22f9965ca9b2e3"
server_key_version: "encryptionkey@0"
I20260808 07:12:03.859052 501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:12:03.859907 515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:03.860081 501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:03.860172 501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "24d2798fe1cc426492745b79c2ed7ed1"
format_stamp: "Formatted at 2026-08-08 07:12:03 on dist-test-slave-6k22"
server_key: "d59c20a9add46694e8bb593e68000283"
server_key_iv: "8d1ee37a21e46135fc22f9965ca9b2e3"
server_key_version: "encryptionkey@0"
I20260808 07:12:03.860239 501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:03.904142 501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:03.904445 501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:03.904559 501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:03.904770 501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:03.905083 501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:03.905126 501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:03.905151 501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:03.905164 501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:03.911254 501 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:38887
I20260808 07:12:03.911348 642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:38887 every 8 connection(s)
I20260808 07:12:03.911707 501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:12:03.913771 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 501
I20260808 07:12:03.913879 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:12:03.914058 22538 external_mini_cluster.cc:1428] Setting key ffb60a8387fe4cbec2917314422a28a9
I20260808 07:12:03.915030 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34651
--builtin_ntp_servers=127.22.2.148:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 07:12:03.917547 477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:03.912448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:36695 (local address 127.22.2.190:34651)
0808 07:12:03.912518 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:03.912519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:03.912796 (+ 277us) server_negotiation.cc:407] Connection header received
0808 07:12:03.913372 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:03.913373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:03.913391 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:03.913459 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:03.914196 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.914710 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.915861 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.916166 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.916247 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:03.917192 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:03.917194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:03.917194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:12:03.917200 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:03.917215 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:12:03.917219 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:03.917299 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:03.917414 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 07:12:03.917460 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:12:03.917771 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:03.912454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34651 (local address 127.22.2.129:36695)
0808 07:12:03.912668 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:03.912704 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:03.913262 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:03.913535 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:03.913538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:03.913546 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:03.914087 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:03.914092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.914828 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.914828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:03.915781 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:03.915784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:03.916275 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:03.916276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:03.916406 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:03.916992 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:12:03.916992 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:03.917025 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:12:03.917049 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:03.917303 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:03.917310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:03.917313 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:12:03.917399 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:12:03.917511 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260808 07:12:03.918274 643 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34651
I20260808 07:12:03.918392 643 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:03.918587 643 heartbeater.cc:511] Master 127.22.2.190:34651 requested a full tablet report, sending...
I20260808 07:12:03.919188 437 ts_manager.cc:194] Registered new tserver with Master: 24d2798fe1cc426492745b79c2ed7ed1 (127.22.2.129:38887)
I20260808 07:12:03.920209 437 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.129:36695
W20260808 07:12:03.998659 646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:03.998857 646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:03.998876 646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:03.999996 646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:04.000036 646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:04.000061 646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:12:04.001309 646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:04.001533 646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:04.001847 646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:04.006937 651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.006913 655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.006989 652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:04.007018 646 server_base.cc:1034] running on GCE node
I20260808 07:12:04.007352 646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:04.007627 646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:04.008790 646 hybrid_clock.cc:648] HybridClock initialized: now 1786173124008769 us; error 36 us; skew 500 ppm
I20260808 07:12:04.009985 646 webserver.cc:460] Webserver started at http://127.22.2.130:40877/ using document root <none> and password file <none>
I20260808 07:12:04.010238 646 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:04.010317 646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:04.010459 646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:04.011603 646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "df72f649f72b4292a1924a0660ed0a70"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "a3a9f48366e742b24d42db153c096f61"
server_key_iv: "256053a7b2454c0bd6b8e5e6938278a3"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.011960 646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "df72f649f72b4292a1924a0660ed0a70"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "a3a9f48366e742b24d42db153c096f61"
server_key_iv: "256053a7b2454c0bd6b8e5e6938278a3"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.013254 646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:12:04.013988 661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.014166 646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:12:04.014269 646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "df72f649f72b4292a1924a0660ed0a70"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "a3a9f48366e742b24d42db153c096f61"
server_key_iv: "256053a7b2454c0bd6b8e5e6938278a3"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.014343 646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:04.044705 646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:04.045043 646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:04.045205 646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:04.045462 646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:04.045809 646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:04.045861 646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.045902 646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:04.045943 646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.051757 646 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:44273
I20260808 07:12:04.051810 792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:44273 every 8 connection(s)
I20260808 07:12:04.052135 646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:12:04.057147 477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.052839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:39295 (local address 127.22.2.190:34651)
0808 07:12:04.052921 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:04.052924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:04.053138 (+ 214us) server_negotiation.cc:407] Connection header received
0808 07:12:04.053618 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:04.053619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:04.053632 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:04.053705 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:04.054333 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:04.054849 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:04.055819 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:04.056012 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:04.056068 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:04.056912 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:04.056913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:04.056913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:12:04.056917 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:04.056930 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:12:04.056933 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:04.056978 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:04.057060 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 07:12:04.057080 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:12:04.057397 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.052794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34651 (local address 127.22.2.130:39295)
0808 07:12:04.053011 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:04.053046 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:04.053531 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:04.053761 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:04.053764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:04.053773 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:04.054250 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:04.054256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:04.054929 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:04.054930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:04.055748 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:04.055750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:04.056086 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:04.056087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:04.056188 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:04.056776 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:12:04.056776 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:04.056812 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:12:04.056834 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:04.056990 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:04.056995 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:04.056997 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:12:04.057048 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:12:04.057125 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260808 07:12:04.057866 793 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34651
I20260808 07:12:04.057965 793 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:04.058141 793 heartbeater.cc:511] Master 127.22.2.190:34651 requested a full tablet report, sending...
I20260808 07:12:04.058542 437 ts_manager.cc:194] Registered new tserver with Master: df72f649f72b4292a1924a0660ed0a70 (127.22.2.130:44273)
I20260808 07:12:04.058954 437 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.130:39295
I20260808 07:12:04.061547 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 646
I20260808 07:12:04.061635 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:12:04.061852 22538 external_mini_cluster.cc:1428] Setting key 8983dea94ccd68986768f13f1623454b
I20260808 07:12:04.062749 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34651
--builtin_ntp_servers=127.22.2.148:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 07:12:04.145102 796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:04.145403 796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:04.145438 796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:04.146701 796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:04.146742 796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:04.146766 796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:12:04.148182 796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:34651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:04.148408 796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:04.148689 796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:04.153757 807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.153764 805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.153779 804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:04.153913 796 server_base.cc:1034] running on GCE node
I20260808 07:12:04.154227 796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:04.154491 796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:04.155623 796 hybrid_clock.cc:648] HybridClock initialized: now 1786173124155609 us; error 40 us; skew 500 ppm
I20260808 07:12:04.156898 796 webserver.cc:460] Webserver started at http://127.22.2.131:40643/ using document root <none> and password file <none>
I20260808 07:12:04.157141 796 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:04.157207 796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:04.157353 796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:04.158475 796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "a4d7a22585c74848a72296df3ed09523"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "579423bc2acabe78449a870c2cbad606"
server_key_iv: "537ede8f26465d03f8f7d2c0a17128ce"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.158849 796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "a4d7a22585c74848a72296df3ed09523"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "579423bc2acabe78449a870c2cbad606"
server_key_iv: "537ede8f26465d03f8f7d2c0a17128ce"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.160252 796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:12:04.161016 813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.161182 796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:12:04.161283 796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "a4d7a22585c74848a72296df3ed09523"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "579423bc2acabe78449a870c2cbad606"
server_key_iv: "537ede8f26465d03f8f7d2c0a17128ce"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.161365 796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:04.172399 796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:04.172727 796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:04.172873 796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:04.173120 796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:04.173522 796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:04.173583 796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.173610 796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:04.173635 796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.180735 796 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:45635
I20260808 07:12:04.180755 940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:45635 every 8 connection(s)
I20260808 07:12:04.181154 796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:12:04.186492 477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.181929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:46949 (local address 127.22.2.190:34651)
0808 07:12:04.181979 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:04.181981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:04.182237 (+ 256us) server_negotiation.cc:407] Connection header received
0808 07:12:04.182802 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:04.182804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:04.182817 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:04.182870 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:04.183592 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:04.184086 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:04.185192 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:04.185402 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:04.185482 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:04.186219 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:04.186221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:04.186221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:12:04.186227 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:04.186243 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:12:04.186248 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:04.186298 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:04.186383 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:12:04.186410 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:12:04.186709 943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.181912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:34651 (local address 127.22.2.131:46949)
0808 07:12:04.182113 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:04.182145 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:04.182689 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:04.182948 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:04.182953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:04.182962 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:04.183479 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:04.183485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:04.184181 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:04.184182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:04.185087 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:04.185090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:04.185490 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:04.185491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:04.185623 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:04.186086 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:12:04.186086 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:04.186112 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:12:04.186134 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:04.186297 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:04.186302 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:04.186305 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:12:04.186385 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:12:04.186486 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260808 07:12:04.187175 941 heartbeater.cc:347] Connected to a master server at 127.22.2.190:34651
I20260808 07:12:04.187255 941 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:04.187489 941 heartbeater.cc:511] Master 127.22.2.190:34651 requested a full tablet report, sending...
I20260808 07:12:04.187897 437 ts_manager.cc:194] Registered new tserver with Master: a4d7a22585c74848a72296df3ed09523 (127.22.2.131:45635)
I20260808 07:12:04.188239 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 796
I20260808 07:12:04.188275 437 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.131:46949
I20260808 07:12:04.188347 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:12:04.188592 22538 external_mini_cluster.cc:1428] Setting key 7dbe099600e094526eb0ad260690fc2c
I20260808 07:12:04.189429 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:12:04.191108 477 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:04.190679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.22.2.190:34651)
0808 07:12:04.190746 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:04.190748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:04.190829 (+ 81us) server_negotiation.cc:407] Connection header received
0808 07:12:04.190871 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:04.190872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:04.190880 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:04.190925 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:04.191049 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56774: BlockingRecv error: recv got EOF from 127.0.0.1:56774 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:12:04.191180 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 501
I20260808 07:12:04.201666 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 646
I20260808 07:12:04.208060 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 796
I20260808 07:12:04.213806 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 402
2026-08-08T07:12:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (653 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T07:12:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:12:04Z Disabled control of system clock
I20260808 07:12:04.242026 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40339
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:41037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:12:04.324577 952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:04.324823 952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:04.324858 952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:04.325974 952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:12:04.326040 952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:04.326062 952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:04.326084 952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:12:04.326112 952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:12:04.327461 952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40339
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:04.327757 952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:04.328054 952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:04.332935 957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.333014 958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.332909 961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:04.333037 952 server_base.cc:1034] running on GCE node
I20260808 07:12:04.333386 952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:04.333665 952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:04.334838 952 hybrid_clock.cc:648] HybridClock initialized: now 1786173124334825 us; error 29 us; skew 500 ppm
I20260808 07:12:04.336207 952 webserver.cc:460] Webserver started at http://127.22.2.190:40617/ using document root <none> and password file <none>
I20260808 07:12:04.336457 952 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:04.336524 952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:04.336664 952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:04.337670 952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "e64e679664304f308c36fb9035cb621a"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "0c4e1e0f61eac0d16456927f5e07e6fc"
server_key_iv: "b8b4fdd54aa204d1a5bc8416b5529e54"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.338012 952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "e64e679664304f308c36fb9035cb621a"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "0c4e1e0f61eac0d16456927f5e07e6fc"
server_key_iv: "b8b4fdd54aa204d1a5bc8416b5529e54"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.339358 952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:12:04.340133 967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.340292 952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.340405 952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "e64e679664304f308c36fb9035cb621a"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "0c4e1e0f61eac0d16456927f5e07e6fc"
server_key_iv: "b8b4fdd54aa204d1a5bc8416b5529e54"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.340495 952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:04.343715 952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:04.343997 952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:04.344152 952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:04.348073 952 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:40339
I20260808 07:12:04.348102 1022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:40339 every 8 connection(s)
I20260808 07:12:04.348413 952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:12:04.349031 1024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:04.351740 1024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a: Bootstrap starting.
I20260808 07:12:04.352300 1024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:04.352602 1024 log.cc:824] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:04.353253 1024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a: No bootstrap required, opened a new log
I20260808 07:12:04.354807 1024 raft_consensus.cc:348] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e679664304f308c36fb9035cb621a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40339 } }
I20260808 07:12:04.354938 1024 raft_consensus.cc:374] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:04.355006 1024 raft_consensus.cc:729] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64e679664304f308c36fb9035cb621a, State: Initialized, Role: FOLLOWER
I20260808 07:12:04.355165 1024 consensus_queue.cc:260] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e679664304f308c36fb9035cb621a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40339 } }
I20260808 07:12:04.355247 1024 raft_consensus.cc:388] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:12:04.355296 1024 raft_consensus.cc:482] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:12:04.355343 1024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:04.355942 1024 raft_consensus.cc:504] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e679664304f308c36fb9035cb621a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40339 } }
I20260808 07:12:04.356050 1024 leader_election.cc:302] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e64e679664304f308c36fb9035cb621a; no voters:
I20260808 07:12:04.356222 1024 leader_election.cc:288] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:12:04.356247 1027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:12:04.356289 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 952
I20260808 07:12:04.356392 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:12:04.356415 1027 raft_consensus.cc:686] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [term 1 LEADER]: Becoming Leader. State: Replica: e64e679664304f308c36fb9035cb621a, State: Running, Role: LEADER
I20260808 07:12:04.356490 1024 sys_catalog.cc:564] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:12:04.356508 1027 consensus_queue.cc:237] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e679664304f308c36fb9035cb621a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40339 } }
I20260808 07:12:04.356665 22538 external_mini_cluster.cc:1428] Setting key 266434254bc0eafb4e7cb855742dccd6
I20260808 07:12:04.356828 1029 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e64e679664304f308c36fb9035cb621a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e679664304f308c36fb9035cb621a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40339 } } }
I20260808 07:12:04.356910 1029 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [sys.catalog]: This master's current role is: LEADER
I20260808 07:12:04.357167 1030 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e64e679664304f308c36fb9035cb621a. Latest consensus state: current_term: 1 leader_uuid: "e64e679664304f308c36fb9035cb621a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64e679664304f308c36fb9035cb621a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40339 } } }
I20260808 07:12:04.357283 1030 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a [sys.catalog]: This master's current role is: LEADER
W20260808 07:12:04.358503 1047 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:12:04.358561 1047 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:12:04.358861 1033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:12:04.359071 1033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:12:04.359328 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.357349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.22.2.190:40339)
0808 07:12:04.357708 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:04.357716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:04.357745 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:12:04.357863 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:04.357874 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:04.357939 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:04.358128 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:04.358831 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:04.358834 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:04.358835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:12:04.358839 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:04.358883 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:12:04.358897 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:04.359010 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:04.359094 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 07:12:04.359146 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":109,"threads_started":1}
I20260808 07:12:04.360632 1033 catalog_manager.cc:1347] Generated new cluster ID: 000868c7021047bb90b8514fa7423094
I20260808 07:12:04.360687 1033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:12:04.372994 1033 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:12:04.373203 1033 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:12:04.380426 1033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e64e679664304f308c36fb9035cb621a: Generated new TSK 0
I20260808 07:12:04.380628 1033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:12:04.390822 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40339
--builtin_ntp_servers=127.22.2.148:41037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:12:04.487603 1050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:04.487846 1050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:04.487885 1050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:04.489076 1050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:04.489137 1050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:04.489181 1050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:12:04.490532 1050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:04.490792 1050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:04.491076 1050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:04.496407 1060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.496407 1063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:04.496594 1050 server_base.cc:1034] running on GCE node
W20260808 07:12:04.496481 1059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:04.497004 1050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:04.497252 1050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:04.498417 1050 hybrid_clock.cc:648] HybridClock initialized: now 1786173124498405 us; error 32 us; skew 500 ppm
I20260808 07:12:04.499737 1050 webserver.cc:460] Webserver started at http://127.22.2.129:46593/ using document root <none> and password file <none>
I20260808 07:12:04.499969 1050 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:04.500033 1050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:04.500188 1050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:04.501298 1050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "a759b28b5d1045e88878afa7c9782af8"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "e34123136ffa88e52efca49c1d53d572"
server_key_iv: "4c955ead17715dfddc06499a9bae8a4f"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.501667 1050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "a759b28b5d1045e88878afa7c9782af8"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "e34123136ffa88e52efca49c1d53d572"
server_key_iv: "4c955ead17715dfddc06499a9bae8a4f"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.503090 1050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:12:04.503955 1070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.504192 1050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:12:04.504324 1050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "a759b28b5d1045e88878afa7c9782af8"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "e34123136ffa88e52efca49c1d53d572"
server_key_iv: "4c955ead17715dfddc06499a9bae8a4f"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.504410 1050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:04.507860 1050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:04.508119 1050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:04.508256 1050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:04.508460 1050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:04.508760 1050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:04.508810 1050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.508849 1050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:04.508876 1050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.514564 1050 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:32941
I20260808 07:12:04.514654 1193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:32941 every 8 connection(s)
I20260808 07:12:04.514952 1050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:12:04.517066 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 1050
I20260808 07:12:04.517217 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:12:04.517439 22538 external_mini_cluster.cc:1428] Setting key c96b093945d0a2cf04d68eb63779ff58
I20260808 07:12:04.518452 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.515787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:36069 (local address 127.22.2.190:40339)
0808 07:12:04.515858 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:04.515860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:04.516015 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:12:04.516462 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:04.516463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:04.516471 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:04.516539 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:04.518144 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:04.518146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:04.518146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:12:04.518152 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:04.518171 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:12:04.518177 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:04.518243 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:04.518364 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 07:12:04.518380 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:12:04.518622 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.515700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40339 (local address 127.22.2.129:36069)
0808 07:12:04.515888 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:04.515929 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:04.516380 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:04.516592 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:04.516595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:04.516603 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:04.517869 (+ 1266us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:12:04.517870 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:04.517907 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:12:04.517943 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:04.518253 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:04.518261 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:04.518265 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:12:04.518366 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:12:04.518425 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260808 07:12:04.518673 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40339
--builtin_ntp_servers=127.22.2.148:41037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 07:12:04.519047 1194 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40339
I20260808 07:12:04.519171 1194 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:04.519341 1194 heartbeater.cc:511] Master 127.22.2.190:40339 requested a full tablet report, sending...
I20260808 07:12:04.519861 985 ts_manager.cc:194] Registered new tserver with Master: a759b28b5d1045e88878afa7c9782af8 (127.22.2.129:32941)
W20260808 07:12:04.600023 1201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:04.600263 1201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:04.600311 1201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:04.601442 1201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:04.601509 1201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:04.601548 1201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:12:04.602864 1201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:04.603123 1201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:04.603430 1201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:04.608377 1206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.608371 1207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.608371 1209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:04.608703 1201 server_base.cc:1034] running on GCE node
I20260808 07:12:04.608911 1201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:04.609138 1201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:04.610286 1201 hybrid_clock.cc:648] HybridClock initialized: now 1786173124610271 us; error 28 us; skew 500 ppm
I20260808 07:12:04.611454 1201 webserver.cc:460] Webserver started at http://127.22.2.130:39213/ using document root <none> and password file <none>
I20260808 07:12:04.611716 1201 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:04.611784 1201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:04.611915 1201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:04.612941 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "a0d7f7d1e93142dcb57d6f2f9a2de2c6"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "b0f6ef73de7b653918bfba809b84e1c1"
server_key_iv: "2f47f917da66df0771e59a567531ee62"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.613299 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "a0d7f7d1e93142dcb57d6f2f9a2de2c6"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "b0f6ef73de7b653918bfba809b84e1c1"
server_key_iv: "2f47f917da66df0771e59a567531ee62"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.614548 1201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:12:04.615224 1215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.615401 1201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:12:04.615495 1201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "a0d7f7d1e93142dcb57d6f2f9a2de2c6"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "b0f6ef73de7b653918bfba809b84e1c1"
server_key_iv: "2f47f917da66df0771e59a567531ee62"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.615599 1201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:04.618818 1201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:04.619065 1201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:04.619203 1201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:04.619412 1201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:04.619763 1201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:04.619823 1201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.619863 1201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:04.619890 1201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.625594 1201 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:37233
I20260808 07:12:04.625640 1340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:37233 every 8 connection(s)
I20260808 07:12:04.625927 1201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:12:04.628748 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.626575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:59817 (local address 127.22.2.190:40339)
0808 07:12:04.626644 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:04.626646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:04.626877 (+ 231us) server_negotiation.cc:407] Connection header received
0808 07:12:04.627308 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:04.627309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:04.627315 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:04.627369 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:04.628547 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:04.628548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:04.628548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:12:04.628551 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:04.628564 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:12:04.628568 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:04.628613 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:04.628684 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:12:04.628696 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:12:04.628947 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.626565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40339 (local address 127.22.2.130:59817)
0808 07:12:04.626764 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:04.626796 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:04.627228 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:04.627426 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:04.627429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:04.627437 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:04.628419 (+ 982us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:12:04.628419 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:04.628445 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:12:04.628473 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:04.628621 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:04.628626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:04.628628 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:12:04.628682 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:12:04.628729 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260808 07:12:04.629387 1342 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40339
I20260808 07:12:04.629485 1342 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:04.629626 1342 heartbeater.cc:511] Master 127.22.2.190:40339 requested a full tablet report, sending...
I20260808 07:12:04.629967 985 ts_manager.cc:194] Registered new tserver with Master: a0d7f7d1e93142dcb57d6f2f9a2de2c6 (127.22.2.130:37233)
I20260808 07:12:04.634014 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 1201
I20260808 07:12:04.634111 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:12:04.634269 22538 external_mini_cluster.cc:1428] Setting key 9adcc559f4514f13329590aab1aecbeb
I20260808 07:12:04.635090 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40339
--builtin_ntp_servers=127.22.2.148:41037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:12:04.717044 1345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:04.717249 1345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:04.717271 1345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:04.718447 1345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:04.718487 1345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:04.718513 1345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:12:04.719986 1345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:40339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:04.720171 1345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:04.720409 1345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:04.725368 1353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.725399 1352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:04.725385 1356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:04.725821 1345 server_base.cc:1034] running on GCE node
I20260808 07:12:04.726058 1345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:04.726310 1345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:04.727466 1345 hybrid_clock.cc:648] HybridClock initialized: now 1786173124727463 us; error 38 us; skew 500 ppm
I20260808 07:12:04.728763 1345 webserver.cc:460] Webserver started at http://127.22.2.131:46199/ using document root <none> and password file <none>
I20260808 07:12:04.729008 1345 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:04.729079 1345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:04.729231 1345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:04.730432 1345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "49ba4a6d6908479e895bdbcd05212baf"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "96eb7d83dc6d31c03d49b96d2e275b02"
server_key_iv: "95513b529c8744c81dbb8add6d18e0ea"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.730794 1345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "49ba4a6d6908479e895bdbcd05212baf"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "96eb7d83dc6d31c03d49b96d2e275b02"
server_key_iv: "95513b529c8744c81dbb8add6d18e0ea"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.732479 1345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:12:04.733178 1363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.733349 1345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:12:04.733448 1345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "49ba4a6d6908479e895bdbcd05212baf"
format_stamp: "Formatted at 2026-08-08 07:12:04 on dist-test-slave-6k22"
server_key: "96eb7d83dc6d31c03d49b96d2e275b02"
server_key_iv: "95513b529c8744c81dbb8add6d18e0ea"
server_key_version: "encryptionkey@0"
I20260808 07:12:04.733520 1345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:04.736862 1345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:04.737109 1345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:04.737251 1345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:04.737457 1345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:04.737756 1345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:04.737811 1345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.737852 1345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:04.737880 1345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:04.743827 1345 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:35927
I20260808 07:12:04.743891 1494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:35927 every 8 connection(s)
I20260808 07:12:04.744243 1345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:12:04.746942 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.744903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:47041 (local address 127.22.2.190:40339)
0808 07:12:04.744962 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:04.744964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:04.745177 (+ 213us) server_negotiation.cc:407] Connection header received
0808 07:12:04.745599 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:04.745600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:04.745606 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:04.745659 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:04.746767 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:04.746767 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:04.746768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:12:04.746770 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:04.746780 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:12:04.746783 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:04.746832 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:04.746877 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 07:12:04.746889 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:12:04.747150 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:04.744896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40339 (local address 127.22.2.131:47041)
0808 07:12:04.745068 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:04.745099 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:04.745520 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:04.745705 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:04.745708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:04.745716 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:04.746643 (+ 927us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:12:04.746643 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:04.746669 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:12:04.746697 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:04.746832 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:04.746835 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:04.746837 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:12:04.746885 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:12:04.746929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260808 07:12:04.747610 1495 heartbeater.cc:347] Connected to a master server at 127.22.2.190:40339
I20260808 07:12:04.747694 1495 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:04.747848 1495 heartbeater.cc:511] Master 127.22.2.190:40339 requested a full tablet report, sending...
I20260808 07:12:04.748183 985 ts_manager.cc:194] Registered new tserver with Master: 49ba4a6d6908479e895bdbcd05212baf (127.22.2.131:35927)
I20260808 07:12:04.750242 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 1345
I20260808 07:12:04.750324 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:12:04.750466 22538 external_mini_cluster.cc:1428] Setting key bcc157a9f6471bea17639347040d7128
I20260808 07:12:04.751237 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:12:04.752995 1040 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:12:04.752530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.22.2.190:40339)
0808 07:12:04.752566 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:04.752568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:04.752690 (+ 122us) server_negotiation.cc:407] Connection header received
0808 07:12:04.752742 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:04.752743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:04.752748 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:04.752792 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:04.752929 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40524: BlockingRecv error: recv got EOF from 127.0.0.1:40524 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:12:04.753013 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 1050
I20260808 07:12:04.766284 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 1201
I20260808 07:12:04.774287 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 1345
I20260808 07:12:04.783972 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 952
2026-08-08T07:12:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (571 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:12:04 dist-test-slave-6k22 krb5kdc[1508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:12:04 dist-test-slave-6k22 krb5kdc[1508](info): set up 2 sockets
Aug 08 07:12:04 dist-test-slave-6k22 krb5kdc[1508](info): commencing operation
krb5kdc: starting...
W20260808 07:12:05.926481 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:12:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:12:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:12:06.050521 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:32851
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:32851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:06.132812 1523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:06.133056 1523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:06.133093 1523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:06.134229 1523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:12:06.134315 1523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:06.134341 1523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:06.134363 1523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:12:06.134389 1523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:12:06.135952 1523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:32851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:32851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:06.136216 1523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:06.136446 1523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:06.141366 1529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:06.141613 1523 server_base.cc:1034] running on GCE node
W20260808 07:12:06.141361 1533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:06.141357 1530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:06.141942 1523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:06.142186 1523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:06.143321 1523 hybrid_clock.cc:648] HybridClock initialized: now 1786173126143307 us; error 30 us; skew 500 ppm
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:06.145763 1523 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:12:06.146205 1523 webserver.cc:460] Webserver started at http://127.22.2.190:36257/ using document root <none> and password file <none>
I20260808 07:12:06.146430 1523 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:06.146477 1523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:06.146631 1523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:06.147758 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "0ecd3db7a4fa40498fd6d67a5b33261a"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "10fe914eb8ae86b259642a9bac731209"
server_key_iv: "33c883244def218ceeeca9755f021579"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.148130 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "0ecd3db7a4fa40498fd6d67a5b33261a"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "10fe914eb8ae86b259642a9bac731209"
server_key_iv: "33c883244def218ceeeca9755f021579"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.149458 1523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:12:06.150154 1540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.150327 1523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:12:06.150417 1523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "0ecd3db7a4fa40498fd6d67a5b33261a"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "10fe914eb8ae86b259642a9bac731209"
server_key_iv: "33c883244def218ceeeca9755f021579"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.150486 1523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:06.165683 1523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:06.167459 1523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:06.167683 1523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:06.171870 1523 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:32851
I20260808 07:12:06.171911 1600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:32851 every 8 connection(s)
I20260808 07:12:06.172261 1523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:12:06.173346 1601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:06.175693 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 1523
I20260808 07:12:06.175770 1601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a: Bootstrap starting.
I20260808 07:12:06.175807 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:12:06.176021 22538 external_mini_cluster.cc:1428] Setting key 3ad4bb649284ac98734e00b186593823
I20260808 07:12:06.176565 1601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:06.176930 1601 log.cc:824] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:06.177770 1601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a: No bootstrap required, opened a new log
I20260808 07:12:06.179952 1601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ecd3db7a4fa40498fd6d67a5b33261a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 32851 } }
I20260808 07:12:06.180145 1601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:06.180209 1601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecd3db7a4fa40498fd6d67a5b33261a, State: Initialized, Role: FOLLOWER
I20260808 07:12:06.180356 1601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ecd3db7a4fa40498fd6d67a5b33261a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 32851 } }
I20260808 07:12:06.180464 1601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:12:06.180521 1601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:12:06.180594 1601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:06.181290 1601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ecd3db7a4fa40498fd6d67a5b33261a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 32851 } }
I20260808 07:12:06.181435 1601 leader_election.cc:302] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ecd3db7a4fa40498fd6d67a5b33261a; no voters:
I20260808 07:12:06.181671 1601 leader_election.cc:288] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:12:06.181957 1601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:12:06.182497 1607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:12:06.182668 1607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [term 1 LEADER]: Becoming Leader. State: Replica: 0ecd3db7a4fa40498fd6d67a5b33261a, State: Running, Role: LEADER
I20260808 07:12:06.182770 1607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ecd3db7a4fa40498fd6d67a5b33261a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 32851 } }
I20260808 07:12:06.183081 1607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ecd3db7a4fa40498fd6d67a5b33261a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ecd3db7a4fa40498fd6d67a5b33261a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 32851 } } }
I20260808 07:12:06.183148 1607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [sys.catalog]: This master's current role is: LEADER
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:12:06.183295 1607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ecd3db7a4fa40498fd6d67a5b33261a. Latest consensus state: current_term: 1 leader_uuid: "0ecd3db7a4fa40498fd6d67a5b33261a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ecd3db7a4fa40498fd6d67a5b33261a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 32851 } } }
I20260808 07:12:06.183758 1607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a [sys.catalog]: This master's current role is: LEADER
W20260808 07:12:06.183897 1624 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:12:06.183944 1624 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:12:06.187995 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.176555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43720 (local address 127.22.2.190:32851)
0808 07:12:06.176765 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.176775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.176810 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:12:06.177476 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.177485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.177555 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.178405 (+ 850us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:06.178504 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.179368 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.180390 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.181004 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.181116 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.184127 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:06.184131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:06.184131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:06.184145 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:06.186141 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.186616 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.186619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.186619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.186742 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.186968 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.186969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.186969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.187274 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:06.187429 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.187549 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.187732 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260808 07:12:06.191835 1626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:12:06.192039 1626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:12:06.193522 1626 catalog_manager.cc:1347] Generated new cluster ID: 99f504f081a54abc9b5b1b60aeb389ef
I20260808 07:12:06.193573 1626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:12:06.205389 1626 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:12:06.205891 1626 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:12:06.213317 1626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a: Generated new TSK 0
I20260808 07:12:06.213498 1626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:12:06.250973 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:32851
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:06.335778 1633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:06.336001 1633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:06.336040 1633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:06.337184 1633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:06.337249 1633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:06.337294 1633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:12:06.338743 1633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:32851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:06.338997 1633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:06.339216 1633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:06.344261 1639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:06.344393 1641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:06.344218 1638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:06.344466 1633 server_base.cc:1034] running on GCE node
I20260808 07:12:06.344767 1633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:06.344961 1633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:06.346094 1633 hybrid_clock.cc:648] HybridClock initialized: now 1786173126346077 us; error 34 us; skew 500 ppm
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:06.348733 1633 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:12:06.349107 1633 webserver.cc:460] Webserver started at http://127.22.2.129:37493/ using document root <none> and password file <none>
I20260808 07:12:06.349283 1633 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:06.349323 1633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:06.349409 1633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:06.350382 1633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "7f254922582740f79e98cffff213a580"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "e6a52898bacd90f994cb0e586a82162d"
server_key_iv: "d4a7e948bd36824a8414ae58636f60ca"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.350677 1633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "7f254922582740f79e98cffff213a580"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "e6a52898bacd90f994cb0e586a82162d"
server_key_iv: "d4a7e948bd36824a8414ae58636f60ca"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.352072 1633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:12:06.352769 1649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.352959 1633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.353075 1633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "7f254922582740f79e98cffff213a580"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "e6a52898bacd90f994cb0e586a82162d"
server_key_iv: "d4a7e948bd36824a8414ae58636f60ca"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.353142 1633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:06.376878 1633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:06.378791 1633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:06.378942 1633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:06.379195 1633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:06.379521 1633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:06.379575 1633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.379640 1633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:06.379678 1633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.385823 1633 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:43705
I20260808 07:12:06.385881 1777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:43705 every 8 connection(s)
I20260808 07:12:06.386207 1633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:12:06.386772 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 1633
I20260808 07:12:06.386890 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:12:06.387127 22538 external_mini_cluster.cc:1428] Setting key cc8f02b290e7bad3bee1247240a83c07
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 07:12:06.397043 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.386900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:33491 (local address 127.22.2.190:32851)
0808 07:12:06.386966 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.386968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.387279 (+ 311us) server_negotiation.cc:407] Connection header received
0808 07:12:06.387872 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.387874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.387886 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.387935 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:06.389283 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.389910 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.390895 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.391229 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.391305 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.393968 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:06.393969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:06.393970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:06.393979 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:06.395618 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.396002 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.396004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.396004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.396073 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.396337 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.396338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.396339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.396492 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:06.396567 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.396829 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.396929 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:12:06.397145 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.386889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:32851 (local address 127.22.2.129:33491)
0808 07:12:06.387080 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.387131 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:06.387795 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:06.388007 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:06.388011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:06.388893 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:06.389181 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.389188 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.390002 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.390003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.390812 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.390814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.391364 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.391365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.391474 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.392116 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:06.392124 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:06.393871 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:06.395695 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:06.395702 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:06.395704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:06.395925 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:06.396153 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:06.396153 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:06.396154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:06.396270 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:06.396561 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:06.396566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:06.396681 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 07:12:06.396774 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 07:12:06.396931 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260808 07:12:06.397686 1778 heartbeater.cc:347] Connected to a master server at 127.22.2.190:32851
I20260808 07:12:06.397794 1778 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:06.398042 1778 heartbeater.cc:511] Master 127.22.2.190:32851 requested a full tablet report, sending...
I20260808 07:12:06.398576 1558 ts_manager.cc:194] Registered new tserver with Master: 7f254922582740f79e98cffff213a580 (127.22.2.129:43705)
I20260808 07:12:06.399174 1558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:33491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:12:06.414438 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:32851
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:06.498672 1783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:06.498904 1783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:06.498942 1783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:06.500123 1783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:06.500182 1783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:06.500227 1783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:12:06.501643 1783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:32851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:06.501894 1783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:06.502115 1783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:06.507058 1791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:06.507160 1789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:06.507061 1788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:06.507678 1783 server_base.cc:1034] running on GCE node
I20260808 07:12:06.507890 1783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:06.508116 1783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:06.509268 1783 hybrid_clock.cc:648] HybridClock initialized: now 1786173126509256 us; error 30 us; skew 500 ppm
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:06.511849 1783 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:12:06.512316 1783 webserver.cc:460] Webserver started at http://127.22.2.130:39817/ using document root <none> and password file <none>
I20260808 07:12:06.512539 1783 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:06.512604 1783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:06.512740 1783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:06.513760 1783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "061e8b3b92ea4f50b287f34616bf4366"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "bd3c3e7ee64449533fa971a0ea1311e3"
server_key_iv: "7977109d1914d0edfc9090c515c7ebae"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.514127 1783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "061e8b3b92ea4f50b287f34616bf4366"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "bd3c3e7ee64449533fa971a0ea1311e3"
server_key_iv: "7977109d1914d0edfc9090c515c7ebae"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.515441 1783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:12:06.516279 1799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.516466 1783 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:12:06.516562 1783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "061e8b3b92ea4f50b287f34616bf4366"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "bd3c3e7ee64449533fa971a0ea1311e3"
server_key_iv: "7977109d1914d0edfc9090c515c7ebae"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.516649 1783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:06.542068 1783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:06.544466 1783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:06.544584 1783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:06.544767 1783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:06.545012 1783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:06.545042 1783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.545064 1783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:06.545077 1783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.550791 1783 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:46237
I20260808 07:12:06.550863 1924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:46237 every 8 connection(s)
I20260808 07:12:06.551178 1783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:12:06.559506 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 1783
I20260808 07:12:06.559636 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:12:06.559870 22538 external_mini_cluster.cc:1428] Setting key 97161454cc6e637915835b8ac0393bc9
I20260808 07:12:06.560913 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.551963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:48931 (local address 127.22.2.190:32851)
0808 07:12:06.552023 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.552025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.552243 (+ 218us) server_negotiation.cc:407] Connection header received
0808 07:12:06.552836 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.552837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.552851 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.552903 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:06.553981 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.554530 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.555530 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.555777 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.555882 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.557981 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:06.557983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:06.557983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:06.557991 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:06.559197 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.559738 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.559740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.559740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.559811 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.560192 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.560193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.560194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.560339 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:06.560392 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.560734 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.560823 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:12:06.561053 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.551935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:32851 (local address 127.22.2.130:48931)
0808 07:12:06.552115 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.552151 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:06.552727 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:06.552972 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:06.552976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:06.553669 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:06.553887 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.553893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.554618 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.554619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.555427 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.555429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.555917 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.555918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.556034 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.556520 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:06.556528 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:06.557872 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:06.559290 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:06.559298 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:06.559300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:06.559644 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:06.559877 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:06.559878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:06.559878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:06.560013 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:06.560390 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:06.560396 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:06.560548 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 07:12:06.560671 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:12:06.560829 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260808 07:12:06.561453 1925 heartbeater.cc:347] Connected to a master server at 127.22.2.190:32851
I20260808 07:12:06.561539 1925 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:06.561729 1925 heartbeater.cc:511] Master 127.22.2.190:32851 requested a full tablet report, sending...
I20260808 07:12:06.562067 1558 ts_manager.cc:194] Registered new tserver with Master: 061e8b3b92ea4f50b287f34616bf4366 (127.22.2.130:46237)
I20260808 07:12:06.562520 1558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:48931
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:12:06.586226 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:32851
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:06.668735 1931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:06.668992 1931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:06.669029 1931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:06.670151 1931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:06.670217 1931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:06.670286 1931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:12:06.671797 1931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:32851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:06.672060 1931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:06.672287 1931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:06.677603 1940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:06.677635 1931 server_base.cc:1034] running on GCE node
W20260808 07:12:06.677733 1937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:06.677744 1936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:06.677984 1931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:06.678241 1931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:06.679409 1931 hybrid_clock.cc:648] HybridClock initialized: now 1786173126679371 us; error 49 us; skew 500 ppm
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:06.682651 1931 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:12:06.683266 1931 webserver.cc:460] Webserver started at http://127.22.2.131:46779/ using document root <none> and password file <none>
I20260808 07:12:06.683692 1931 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:06.683769 1931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:06.683915 1931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:12:06.685463 1931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "a52ee33b6bae420f97690df3f03c79f2"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "d805a6bd027de99c9fdb262a48c18993"
server_key_iv: "b4b865f038ca630965a2ce77cabec43f"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.685846 1931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "a52ee33b6bae420f97690df3f03c79f2"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "d805a6bd027de99c9fdb262a48c18993"
server_key_iv: "b4b865f038ca630965a2ce77cabec43f"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.687618 1931 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260808 07:12:06.688905 1947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.689106 1931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:12:06.689206 1931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "a52ee33b6bae420f97690df3f03c79f2"
format_stamp: "Formatted at 2026-08-08 07:12:06 on dist-test-slave-6k22"
server_key: "d805a6bd027de99c9fdb262a48c18993"
server_key_iv: "b4b865f038ca630965a2ce77cabec43f"
server_key_version: "encryptionkey@0"
I20260808 07:12:06.689292 1931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:06.703343 1931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:06.705937 1931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:06.706084 1931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:06.706372 1931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:06.706737 1931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:06.706787 1931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.706820 1931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:06.706846 1931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:06.715044 1931 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:39723
I20260808 07:12:06.715085 2076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:39723 every 8 connection(s)
I20260808 07:12:06.715508 1931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:12:06.721439 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 1931
I20260808 07:12:06.721539 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:12:06.721760 22538 external_mini_cluster.cc:1428] Setting key f22f8c972857c3b6b5f10c0062eba3b9
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[1508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173126, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:12:06.727767 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.716447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:54115 (local address 127.22.2.190:32851)
0808 07:12:06.716505 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.716507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.716547 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:12:06.717348 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.717349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.717360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.717421 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:06.718849 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.719671 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.721312 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.721610 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.721686 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.724378 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:06.724380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:06.724380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:06.724388 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:06.726076 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.726598 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.726599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.726600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.726658 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.726952 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.726953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.726953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.727129 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:06.727198 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.727534 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.727642 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:12:06.728221 2079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.716245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:32851 (local address 127.22.2.131:54115)
0808 07:12:06.716414 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.716465 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:06.717218 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:06.717523 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:06.717528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:06.718405 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:06.718756 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.718763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.719805 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.719806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.721218 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.721222 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.721773 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.721774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.721918 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.722586 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:06.722595 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:06.724280 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:06.726163 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:06.726170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:06.726172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:06.726523 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:06.726724 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:06.726725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:06.726726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:06.726878 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:06.727199 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:06.727205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:06.727381 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 07:12:06.727506 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:12:06.727717 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260808 07:12:06.728724 2077 heartbeater.cc:347] Connected to a master server at 127.22.2.190:32851
I20260808 07:12:06.728835 2077 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:06.729022 2077 heartbeater.cc:511] Master 127.22.2.190:32851 requested a full tablet report, sending...
I20260808 07:12:06.729401 1558 ts_manager.cc:194] Registered new tserver with Master: a52ee33b6bae420f97690df3f03c79f2 (127.22.2.131:39723)
I20260808 07:12:06.729754 1558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:54115
I20260808 07:12:06.733479 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:12:06.740442 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.735216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.22.2.190:32851)
0808 07:12:06.735268 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.735269 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.735367 (+ 98us) server_negotiation.cc:407] Connection header received
0808 07:12:06.735405 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.735406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.735413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.735458 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:06.736037 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.736477 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.737123 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.737329 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.737376 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.738440 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:06.738440 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:06.738441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:06.738446 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:06.739539 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.739862 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.739862 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.739863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.739911 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:06.740059 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:06.740059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:06.740059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:06.740177 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:06.740242 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.740334 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.740387 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:12:06.741779 1558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:12:06.743378 1558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:12:06.749307 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.745561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43705 (local address 127.0.0.1:60174)
0808 07:12:06.745946 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.745958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:06.746012 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:06.746318 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:06.746321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:06.746323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:06.746412 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.746414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.747331 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.747332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.748233 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.748236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.748987 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.748988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.749073 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.749092 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 07:12:06.749171 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:12:06.749225 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":17,"thread_start_us":78,"threads_started":1}
I20260808 07:12:06.749547 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.745846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60174 (local address 127.22.2.129:43705)
0808 07:12:06.746063 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.746068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.746080 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:12:06.746136 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.746138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.746156 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.746395 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:06.746475 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.747229 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.748335 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.748901 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.748987 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.749048 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.749421 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.749468 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260808 07:12:06.750161 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.745561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:46237 (local address 127.0.0.1:43696)
0808 07:12:06.745892 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.745935 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:06.746031 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:06.746319 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:06.746322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:06.746325 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:06.746501 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.746502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.747846 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.747847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.749063 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.749065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.749924 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.749925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.749979 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.749983 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:12:06.750054 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:12:06.750096 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":54,"thread_start_us":53,"threads_started":1}
I20260808 07:12:06.750370 2092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.745685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43696 (local address 127.22.2.130:46237)
0808 07:12:06.745840 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.745845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.746005 (+ 160us) server_negotiation.cc:407] Connection header received
0808 07:12:06.746130 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.746133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.746150 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.746204 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:06.746588 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.747718 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.749161 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.749846 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.749922 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.749976 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.750077 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.750119 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260808 07:12:06.750284 1703 tablet_service.cc:1467] Processing CreateTablet for tablet 009cc6ccdf4944f4a95fd657ea39b69c (DEFAULT_TABLE table=test-table [id=c0c108c598d24bd1ac0e2c54c4b95927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:06.750665 1703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 009cc6ccdf4944f4a95fd657ea39b69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:06.751112 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.745814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:39723 (local address 127.0.0.1:49154)
0808 07:12:06.746176 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.746188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:06.746246 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:06.746479 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:06.746480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:06.746482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:06.746514 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.746515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.747836 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.747837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.749062 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.749065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.750871 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.750872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.750932 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.750936 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:12:06.750993 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:12:06.751044 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":52,"thread_start_us":100,"threads_started":1}
I20260808 07:12:06.751027 1855 tablet_service.cc:1467] Processing CreateTablet for tablet 009cc6ccdf4944f4a95fd657ea39b69c (DEFAULT_TABLE table=test-table [id=c0c108c598d24bd1ac0e2c54c4b95927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:06.751377 1855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 009cc6ccdf4944f4a95fd657ea39b69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:06.751140 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.746027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.22.2.131:39723)
0808 07:12:06.746182 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.746186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.746258 (+ 72us) server_negotiation.cc:407] Connection header received
0808 07:12:06.746318 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.746320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.746333 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.746407 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:06.746588 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.747728 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.750022 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.750694 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.750776 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.750833 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.751023 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.751070 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260808 07:12:06.752130 2004 tablet_service.cc:1467] Processing CreateTablet for tablet 009cc6ccdf4944f4a95fd657ea39b69c (DEFAULT_TABLE table=test-table [id=c0c108c598d24bd1ac0e2c54c4b95927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:06.752444 2004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 009cc6ccdf4944f4a95fd657ea39b69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:06.754325 2097 tablet_bootstrap.cc:492] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Bootstrap starting.
I20260808 07:12:06.754256 2096 tablet_bootstrap.cc:492] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580: Bootstrap starting.
I20260808 07:12:06.755028 2096 tablet_bootstrap.cc:654] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:06.755064 2097 tablet_bootstrap.cc:654] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:06.755635 2096 log.cc:824] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:06.755709 2097 log.cc:824] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:06.756059 2098 tablet_bootstrap.cc:492] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: Bootstrap starting.
I20260808 07:12:06.756457 2096 tablet_bootstrap.cc:492] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580: No bootstrap required, opened a new log
I20260808 07:12:06.756529 2096 ts_tablet_manager.cc:1397] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 07:12:06.756675 2098 tablet_bootstrap.cc:654] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:06.757200 2098 log.cc:824] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:06.757916 2098 tablet_bootstrap.cc:492] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: No bootstrap required, opened a new log
I20260808 07:12:06.757987 2098 ts_tablet_manager.cc:1397] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:12:06.758654 2096 raft_consensus.cc:348] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.758826 2096 raft_consensus.cc:374] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:06.758880 2096 raft_consensus.cc:729] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f254922582740f79e98cffff213a580, State: Initialized, Role: FOLLOWER
I20260808 07:12:06.758996 2096 consensus_queue.cc:260] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.759275 1778 heartbeater.cc:503] Master 127.22.2.190:32851 was elected leader, sending a full tablet report...
I20260808 07:12:06.759647 2097 tablet_bootstrap.cc:492] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: No bootstrap required, opened a new log
I20260808 07:12:06.759712 2097 ts_tablet_manager.cc:1397] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 07:12:06.759285 2096 ts_tablet_manager.cc:1428] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:12:06.760033 2098 raft_consensus.cc:348] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.760185 2098 raft_consensus.cc:374] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:06.760217 2098 raft_consensus.cc:729] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52ee33b6bae420f97690df3f03c79f2, State: Initialized, Role: FOLLOWER
I20260808 07:12:06.760342 2098 consensus_queue.cc:260] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.760613 2098 ts_tablet_manager.cc:1428] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:12:06.760622 2077 heartbeater.cc:503] Master 127.22.2.190:32851 was elected leader, sending a full tablet report...
I20260808 07:12:06.761709 2097 raft_consensus.cc:348] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.761832 2097 raft_consensus.cc:374] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:06.761853 2097 raft_consensus.cc:729] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 061e8b3b92ea4f50b287f34616bf4366, State: Initialized, Role: FOLLOWER
I20260808 07:12:06.761948 2097 consensus_queue.cc:260] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.762204 2097 ts_tablet_manager.cc:1428] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:12:06.762302 1925 heartbeater.cc:503] Master 127.22.2.190:32851 was elected leader, sending a full tablet report...
W20260808 07:12:06.801745 1927 tablet.cc:2369] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:12:06.815840 2106 raft_consensus.cc:482] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:12:06.816025 2106 raft_consensus.cc:504] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.816457 2106 leader_election.cc:288] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 061e8b3b92ea4f50b287f34616bf4366 (127.22.2.130:46237), 7f254922582740f79e98cffff213a580 (127.22.2.129:43705)
I20260808 07:12:06.819628 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.816791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:53547 (local address 127.22.2.129:43705)
0808 07:12:06.816837 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.816839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.816951 (+ 112us) server_negotiation.cc:407] Connection header received
0808 07:12:06.816987 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.816988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.817000 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.817067 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:06.817234 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.817674 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.818657 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.819253 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.819342 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.819352 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.819490 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.819533 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:12:06.819636 2108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.816647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43705 (local address 127.22.2.131:53547)
0808 07:12:06.816884 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.816894 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:06.816943 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:06.817076 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:06.817077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:06.817078 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:06.817162 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.817164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.817796 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.817797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.818571 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.818572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.819341 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.819342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.819420 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.819424 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:12:06.819467 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:12:06.819521 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":15,"thread_start_us":125,"threads_started":1}
I20260808 07:12:06.819655 2092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.816776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:57677 (local address 127.22.2.130:46237)
0808 07:12:06.816825 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.816827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.816835 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:12:06.816867 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.816868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.816881 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.816944 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:06.817151 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.817655 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.818678 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.819254 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.819349 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.819357 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.819529 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.819590 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:12:06.819643 2079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.816638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:46237 (local address 127.22.2.131:57677)
0808 07:12:06.816692 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.816705 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:06.816758 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:06.816974 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:06.816975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:06.816977 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:06.817069 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.817072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.817736 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.817737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.818571 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:06.818574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.819350 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.819352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:06.819440 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.819443 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:12:06.819510 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:12:06.819547 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":42}
I20260808 07:12:06.820042 1876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "009cc6ccdf4944f4a95fd657ea39b69c" candidate_uuid: "a52ee33b6bae420f97690df3f03c79f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "061e8b3b92ea4f50b287f34616bf4366" is_pre_election: true
I20260808 07:12:06.820120 1725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "009cc6ccdf4944f4a95fd657ea39b69c" candidate_uuid: "a52ee33b6bae420f97690df3f03c79f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f254922582740f79e98cffff213a580" is_pre_election: true
I20260808 07:12:06.820221 1876 raft_consensus.cc:2454] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52ee33b6bae420f97690df3f03c79f2 in term 0.
I20260808 07:12:06.820286 1725 raft_consensus.cc:2454] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52ee33b6bae420f97690df3f03c79f2 in term 0.
I20260808 07:12:06.820472 1949 leader_election.cc:302] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 061e8b3b92ea4f50b287f34616bf4366, a52ee33b6bae420f97690df3f03c79f2; no voters:
I20260808 07:12:06.820608 2106 raft_consensus.cc:2781] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:12:06.820667 2106 raft_consensus.cc:482] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:12:06.820690 2106 raft_consensus.cc:3037] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:06.821382 2106 raft_consensus.cc:504] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.821517 2106 leader_election.cc:288] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [CANDIDATE]: Term 1 election: Requested vote from peers 061e8b3b92ea4f50b287f34616bf4366 (127.22.2.130:46237), 7f254922582740f79e98cffff213a580 (127.22.2.129:43705)
I20260808 07:12:06.821671 1725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "009cc6ccdf4944f4a95fd657ea39b69c" candidate_uuid: "a52ee33b6bae420f97690df3f03c79f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f254922582740f79e98cffff213a580"
I20260808 07:12:06.821771 1725 raft_consensus.cc:3037] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:06.822508 1725 raft_consensus.cc:2454] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52ee33b6bae420f97690df3f03c79f2 in term 1.
I20260808 07:12:06.822628 1876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "009cc6ccdf4944f4a95fd657ea39b69c" candidate_uuid: "a52ee33b6bae420f97690df3f03c79f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "061e8b3b92ea4f50b287f34616bf4366"
I20260808 07:12:06.822691 1951 leader_election.cc:302] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f254922582740f79e98cffff213a580, a52ee33b6bae420f97690df3f03c79f2; no voters:
I20260808 07:12:06.822718 1876 raft_consensus.cc:3037] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:06.822790 2106 raft_consensus.cc:2781] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:12:06.822950 2106 raft_consensus.cc:686] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 1 LEADER]: Becoming Leader. State: Replica: a52ee33b6bae420f97690df3f03c79f2, State: Running, Role: LEADER
I20260808 07:12:06.823056 2106 consensus_queue.cc:237] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } }
I20260808 07:12:06.823624 1876 raft_consensus.cc:2454] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52ee33b6bae420f97690df3f03c79f2 in term 1.
I20260808 07:12:06.823911 1558 catalog_manager.cc:5495] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a52ee33b6bae420f97690df3f03c79f2 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "a52ee33b6bae420f97690df3f03c79f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a52ee33b6bae420f97690df3f03c79f2" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39723 } health_report { overall_health: HEALTHY } } }
I20260808 07:12:06.841931 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.839734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.22.2.131:39723)
0808 07:12:06.839794 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:06.839796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:06.839806 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:12:06.839848 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:06.839848 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:06.839868 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:06.839928 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:12:06.840079 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.840542 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.841279 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:06.841461 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:06.841508 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:06.841652 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:12:06.841747 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:12:06.841805 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:06.841842 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:12:06.841859 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:12:06.843814 1725 raft_consensus.cc:1261] T 009cc6ccdf4944f4a95fd657ea39b69c P 7f254922582740f79e98cffff213a580 [term 1 FOLLOWER]: Refusing update from remote peer a52ee33b6bae420f97690df3f03c79f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:12:06.843832 1876 raft_consensus.cc:1261] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 1 FOLLOWER]: Refusing update from remote peer a52ee33b6bae420f97690df3f03c79f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:12:06.844192 2109 consensus_queue.cc:1035] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f254922582740f79e98cffff213a580" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 43705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:12:06.844334 2106 consensus_queue.cc:1035] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "061e8b3b92ea4f50b287f34616bf4366" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:12:06.846822 2113 mvcc.cc:205] Tried to move back new op lower bound from 7316165127549452288 to 7316165127467900928. Current Snapshot: MvccSnapshot[applied={T|T < 7316165127549452288}]
I20260808 07:12:06.849411 1556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:12:06.849510 1556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:12:06.850250 1556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ecd3db7a4fa40498fd6d67a5b33261a: Sending DeleteTablet for 3 replicas of tablet 009cc6ccdf4944f4a95fd657ea39b69c
I20260808 07:12:06.850605 1855 tablet_service.cc:1514] Processing DeleteTablet for tablet 009cc6ccdf4944f4a95fd657ea39b69c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:12:06 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:43696
I20260808 07:12:06.850600 1703 tablet_service.cc:1514] Processing DeleteTablet for tablet 009cc6ccdf4944f4a95fd657ea39b69c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:12:06 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60174
I20260808 07:12:06.850826 2004 tablet_service.cc:1514] Processing DeleteTablet for tablet 009cc6ccdf4944f4a95fd657ea39b69c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:12:06 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:49154
I20260808 07:12:06.850997 2125 tablet_replica.cc:330] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: stopping tablet replica
I20260808 07:12:06.851038 2126 tablet_replica.cc:330] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: stopping tablet replica
I20260808 07:12:06.851135 2126 raft_consensus.cc:2229] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:12:06.851150 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 1633
I20260808 07:12:06.851276 2126 raft_consensus.cc:2258] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:12:06.852926 2126 ts_tablet_manager.cc:1905] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:12:06.853127 2125 raft_consensus.cc:2229] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:12:06.853245 2125 raft_consensus.cc:2258] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:12:06.853688 2125 ts_tablet_manager.cc:1905] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:12:06.855357 2125 ts_tablet_manager.cc:1918] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:12:06.855475 2125 log.cc:1196] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/009cc6ccdf4944f4a95fd657ea39b69c
I20260808 07:12:06.855630 2126 ts_tablet_manager.cc:1918] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:12:06.855715 2126 log.cc:1196] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/009cc6ccdf4944f4a95fd657ea39b69c
I20260808 07:12:06.855998 2125 ts_tablet_manager.cc:1939] T 009cc6ccdf4944f4a95fd657ea39b69c P 061e8b3b92ea4f50b287f34616bf4366: Deleting consensus metadata
I20260808 07:12:06.856038 2126 ts_tablet_manager.cc:1939] T 009cc6ccdf4944f4a95fd657ea39b69c P a52ee33b6bae420f97690df3f03c79f2: Deleting consensus metadata
I20260808 07:12:06.856426 1542 catalog_manager.cc:4841] TS a52ee33b6bae420f97690df3f03c79f2 (127.22.2.131:39723): tablet 009cc6ccdf4944f4a95fd657ea39b69c (table test-table [id=c0c108c598d24bd1ac0e2c54c4b95927]) successfully deleted
I20260808 07:12:06.856524 1542 catalog_manager.cc:4841] TS 061e8b3b92ea4f50b287f34616bf4366 (127.22.2.130:46237): tablet 009cc6ccdf4944f4a95fd657ea39b69c (table test-table [id=c0c108c598d24bd1ac0e2c54c4b95927]) successfully deleted
W20260808 07:12:06.859985 1544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:43705 (error 108)
I20260808 07:12:06.860524 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:06.860387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:43705 (local address 127.0.0.1:60182)
0808 07:12:06.860444 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:06.860485 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:43705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:12:06.860589 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 1783
W20260808 07:12:06.863772 1544 catalog_manager.cc:4584] TS 7f254922582740f79e98cffff213a580 (127.22.2.129:43705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 009cc6ccdf4944f4a95fd657ea39b69c: Network error: Client connection negotiation failed: client connection to 127.22.2.129:43705: connect: Connection refused (error 111)
I20260808 07:12:06.867940 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 1931
I20260808 07:12:06.875118 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 1523
2026-08-08T07:12:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2090 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[2131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[2131](info): set up 2 sockets
Aug 08 07:12:06 dist-test-slave-6k22 krb5kdc[2131](info): commencing operation
krb5kdc: starting...
W20260808 07:12:08.004163 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:12:08 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:12:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:12:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/rangeradmin_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/rangeradmin_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/rangerlookup_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/rangerlookup_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
I20260808 07:12:08.170771 22538 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskyv8zyi/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/postgres -l logfile start
2026-08-08 07:12:10.783 UTC [2165] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:12:10.784 UTC [2165] LOG: listening on IPv4 address "127.22.2.148", port 54999
2026-08-08 07:12:10.787 UTC [2165] LOG: listening on Unix socket "/tmp/.s.PGSQL.54999"
2026-08-08 07:12:10.809 UTC [2169] LOG: database system was shut down at 2026-08-08 07:12:09 UTC
2026-08-08 07:12:10.812 UTC [2165] LOG: database system is ready to accept connections
I20260808 07:12:11.823457 22538 mini_postgres.cc:96] Postgres bound to 54999
2026-08-08 07:12:11.831 UTC [2177] FATAL: database "slave" does not exist
127.22.2.148:54999 - accepting connections
I20260808 07:12:11.832063 22538 mini_ranger.cc:161] Starting Ranger...
I20260808 07:12:11.848055 22538 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:12:12.345655 22538 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:12:12.345755 22538 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-admin
2026-08-08 07:12:12,666 [I] DB FLAVOR :POSTGRES
2026-08-08 07:12:12,667 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:12:12,667 [I] Checking connection..
2026-08-08 07:12:12,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:12:12,926 [I] Checking connection passed.
2026-08-08 07:12:12,926 [I] --------- Verifying version history table ---------
2026-08-08 07:12:12,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:12:13,180 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:12:13,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:12:13,456 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:12:13,456 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:12:13,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:12:13.716 UTC [2249] WARNING: there is no transaction in progress
2026-08-08 07:12:13,723 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:12:13,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:12:13,957 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:12:13,957 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:12:13,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:12:14,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:12:14,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-08 07:12:14,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:12:15,011 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:12:15,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:12:15,266 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:12:15,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:12:15,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:12:15,778 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:12:15,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:12:16.817 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.834 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.850 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.865 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.880 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.900 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.953 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.960 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.975 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.985 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:16.995 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:17.009 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:17.019 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:17.025 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:17.032 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:17.038 UTC [2372] WARNING: there is no transaction in progress
2026-08-08 07:12:17,677 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:12:17,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:12:17,935 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:12:17,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:12:18,178 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:12:18,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:12:18,447 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:12:18,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:12:18,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:12:18,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-08 07:12:19,223 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:12:19,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:12:19,470 [I] DB_PATCHES have already been applied
I20260808 07:12:19.475934 22538 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-admin:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskyv8zyi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:12:19.475997 22538 mini_ranger.cc:193] Using host: 127.22.2.148
I20260808 07:12:19.479534 22538 mini_ranger.cc:227] Ranger admin URL: http://127.22.2.148:33707
Aug 08, 2026 7:12:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:12:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:12:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:12:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:12:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:12:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:12:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:12:20.703181 22538 mini_ranger.cc:160] Time spent starting Ranger: real 8.871s user 0.000s sys 0.011s
Aug 08, 2026 7:12:39 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:12:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 7:12:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:12:39 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:12:40 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:12:41.561134 22538 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/rangerkms_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/rangerkms_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.2.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.2.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:12:41 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173161, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:12:41.646330 22538 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:12:41.656561 22538 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:12:42.540745 22538 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:12:42.540841 22538 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms
I20260808 07:12:42.540861 22538 mini_ranger_kms.cc:227] Using postgres at 127.22.2.148:54999
2026-08-08 07:12:42,814 [I] DB FLAVOR :POSTGRES
2026-08-08 07:12:42,815 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:12:42,815 [I] Checking connection
2026-08-08 07:12:42,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:12:43,108 [I] connection success
2026-08-08 07:12:43,108 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:12:43,108 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:12:43,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:12:43,428 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:12:43,429 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:12:43,429 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:12:43,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:54999/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:12:43,717 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:12:44.146672 22538 mini_ranger_kms.cc:310] Created kms service
I20260808 07:12:44.344204 22538 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:12:44.386807 22538 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:12:44 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173161, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:12:44.794759 22538 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:12:44.794883 22538 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms:/tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:12:44.794904 22538 mini_ranger_kms.cc:240] Using host: 127.22.2.148
I20260808 07:12:44.799546 22538 mini_ranger_kms.cc:276] Ranger KMS PID: 2583
I20260808 07:12:44.799710 22538 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.2.148:54545
Aug 08, 2026 7:12:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:12:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:12:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:12:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:12:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:12:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 07:12:47 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173167, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:47.192874 22538 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.547s user 0.005s sys 0.253s
I20260808 07:12:47.192986 22538 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:12:47.193073 22538 mini_ranger_kms.cc:401] 127.22.2.148:54545/kms/v1/keys
Aug 08 07:12:47 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173167, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
Aug 08 07:12:47 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173167, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
Aug 08 07:12:50 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:12:50.551276 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42779
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42779
--ranger_config_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:54545/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:50.675074 2641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:50.675316 2641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:50.675349 2641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:50.676717 2641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:12:50.676766 2641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:50.676788 2641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:50.676800 2641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:12:50.676812 2641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:12:50.678838 2641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.2.148:54545/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42779
--ranger_config_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:50.679111 2641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:50.679361 2641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:50.685817 2648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:50.685848 2654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:50.686040 2647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:50.686242 2641 server_base.cc:1034] running on GCE node
I20260808 07:12:50.686513 2641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:50.686867 2641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:50.688055 2641 hybrid_clock.cc:648] HybridClock initialized: now 1786173170688017 us; error 53 us; skew 500 ppm
Aug 08 07:12:50 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173170, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:50.691919 2641 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:12:50.692504 2641 webserver.cc:460] Webserver started at http://127.22.2.190:44705/ using document root <none> and password file <none>
I20260808 07:12:50.692792 2641 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:50.692857 2641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:50.693019 2641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:12:50 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173170, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:12:50.743647 2641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "c8bf34c3b28c440f85b7919fa549588c"
format_stamp: "Formatted at 2026-08-08 07:12:50 on dist-test-slave-6k22"
server_key: "8beda813d89427bf001d2ce38f2bc391"
server_key_iv: "70c4a1d9cc14ab21cc8ec2d7b737a41e"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:50.744138 2641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "c8bf34c3b28c440f85b7919fa549588c"
format_stamp: "Formatted at 2026-08-08 07:12:50 on dist-test-slave-6k22"
server_key: "8beda813d89427bf001d2ce38f2bc391"
server_key_iv: "70c4a1d9cc14ab21cc8ec2d7b737a41e"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:50.746055 2641 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:12:50.763343 2661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:50.763594 2641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:12:50.763720 2641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "c8bf34c3b28c440f85b7919fa549588c"
format_stamp: "Formatted at 2026-08-08 07:12:50 on dist-test-slave-6k22"
server_key: "8beda813d89427bf001d2ce38f2bc391"
server_key_iv: "70c4a1d9cc14ab21cc8ec2d7b737a41e"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:50.763837 2641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:50.779716 2641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:50.782023 2641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:50.782269 2641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:50.794872 2641 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42779
I20260808 07:12:50.794974 2717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42779 every 8 connection(s)
I20260808 07:12:50.795452 2641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:12:50.796708 2718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:50.798928 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 2641
I20260808 07:12:50.799059 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:12:50.799273 22538 external_mini_cluster.cc:1428] Setting key a1c78239f2be0d952a3706c9a501e9bb
I20260808 07:12:50.801213 2718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c: Bootstrap starting.
I20260808 07:12:50.802520 2718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:50.802873 2718 log.cc:824] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:50.807026 2718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c: No bootstrap required, opened a new log
I20260808 07:12:50.809675 2718 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8bf34c3b28c440f85b7919fa549588c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42779 } }
I20260808 07:12:50.809840 2718 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:50.809886 2718 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8bf34c3b28c440f85b7919fa549588c, State: Initialized, Role: FOLLOWER
I20260808 07:12:50.810030 2718 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8bf34c3b28c440f85b7919fa549588c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42779 } }
I20260808 07:12:50.810112 2718 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:12:50.810173 2718 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:12:50.810230 2718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:50.811188 2718 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8bf34c3b28c440f85b7919fa549588c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42779 } }
I20260808 07:12:50.811326 2718 leader_election.cc:302] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8bf34c3b28c440f85b7919fa549588c; no voters:
I20260808 07:12:50.811517 2718 leader_election.cc:288] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:12:50.811784 2723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:12:50.811868 2718 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 07:12:50 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:12:50.812996 2723 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [term 1 LEADER]: Becoming Leader. State: Replica: c8bf34c3b28c440f85b7919fa549588c, State: Running, Role: LEADER
I20260808 07:12:50.813151 2723 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8bf34c3b28c440f85b7919fa549588c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42779 } }
I20260808 07:12:50.815454 2724 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8bf34c3b28c440f85b7919fa549588c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8bf34c3b28c440f85b7919fa549588c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42779 } } }
I20260808 07:12:50.815591 2724 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [sys.catalog]: This master's current role is: LEADER
I20260808 07:12:50.816013 2730 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8bf34c3b28c440f85b7919fa549588c. Latest consensus state: current_term: 1 leader_uuid: "c8bf34c3b28c440f85b7919fa549588c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8bf34c3b28c440f85b7919fa549588c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42779 } } }
I20260808 07:12:50.816094 2730 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c [sys.catalog]: This master's current role is: LEADER
I20260808 07:12:50.816176 2739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:12:50.816316 2718 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:12:50.816928 2739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:12:50.825148 2721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:50.799851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.22.2.190:42779)
0808 07:12:50.803320 (+ 3469us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:50.803330 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:50.803398 (+ 68us) server_negotiation.cc:407] Connection header received
0808 07:12:50.804348 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:50.804357 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:50.804425 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:50.804585 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:50.805793 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:50.807418 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:50.808609 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:50.809934 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:50.810052 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:50.813522 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:50.813526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:50.813526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:50.813539 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:50.816834 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:50.817301 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:50.817304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:50.817305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:50.817425 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:50.817660 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:50.819044 (+ 1384us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:50.819045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:50.821758 (+ 2713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:50.823253 (+ 1495us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:50.823455 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 07:12:50.824705 (+ 1250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3387,"thread_start_us":141,"threads_started":1}
I20260808 07:12:50.832664 2739 catalog_manager.cc:1347] Generated new cluster ID: 56702ba4700340aa9a3cf8e7b8e5cbaa
I20260808 07:12:50.832803 2739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:12:50.843791 2739 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:12:50.844689 2739 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:12:50.852203 2739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c: Generated new TSK 0
I20260808 07:12:50.852486 2739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:12:51.807448 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:54545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42779
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:51.976734 2766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:51.977029 2766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:51.977070 2766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:51.979010 2766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:51.979079 2766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:51.979123 2766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:12:51.981585 2766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.2.148:54545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:51.981873 2766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:51.982208 2766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:51.989328 2772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:51.989661 2774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:51.989709 2766 server_base.cc:1034] running on GCE node
W20260808 07:12:51.989437 2771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:51.990139 2766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:51.990444 2766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:51.991631 2766 hybrid_clock.cc:648] HybridClock initialized: now 1786173171991597 us; error 48 us; skew 500 ppm
Aug 08 07:12:51 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173171, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:52.001091 2766 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:12:52.001641 2766 webserver.cc:460] Webserver started at http://127.22.2.129:36545/ using document root <none> and password file <none>
I20260808 07:12:52.001969 2766 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:52.002038 2766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:52.002173 2766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173171, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:12:52.026887 2766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "799dae90dfb7452c9f2b14739997ab60"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "5488d976ea8d8ea5ae77cd7f2f5f97d1"
server_key_iv: "17fa8c1be82e1f27a084f66515f9d6b6"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.027448 2766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "799dae90dfb7452c9f2b14739997ab60"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "5488d976ea8d8ea5ae77cd7f2f5f97d1"
server_key_iv: "17fa8c1be82e1f27a084f66515f9d6b6"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.029955 2766 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:12:52.044171 2781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.044457 2766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:12:52.044610 2766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "799dae90dfb7452c9f2b14739997ab60"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "5488d976ea8d8ea5ae77cd7f2f5f97d1"
server_key_iv: "17fa8c1be82e1f27a084f66515f9d6b6"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.044742 2766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:52.061846 2766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:52.064386 2766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:52.064582 2766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:52.064936 2766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:52.065452 2766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:52.065521 2766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.065567 2766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:52.065632 2766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.080839 2766 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:32881
I20260808 07:12:52.080938 2913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:32881 every 8 connection(s)
I20260808 07:12:52.081306 2766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:12:52.081570 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 2766
I20260808 07:12:52.081743 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:12:52.082048 22538 external_mini_cluster.cc:1428] Setting key 7ea2f35cc0a7a48f845de7550575bdfb
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173171, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:12:52.102319 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.083419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42779 (local address 127.22.2.129:60171)
0808 07:12:52.087958 (+ 4539us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.088015 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:52.089286 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:52.089877 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:52.089882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:52.090290 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:52.090710 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.090719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.091817 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.091818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.093273 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.093276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.093802 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.093803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.094047 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.094793 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:52.094802 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:52.098197 (+ 3395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:52.100634 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:52.100640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:52.100642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:52.100915 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:52.101160 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:52.101161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:52.101161 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:52.101274 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:52.101547 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:52.101553 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:52.101715 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 07:12:52.101830 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:12:52.101988 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4469,"thread_start_us":168,"threads_started":1}
I20260808 07:12:52.102819 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.083808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:60171 (local address 127.22.2.190:42779)
0808 07:12:52.089512 (+ 5704us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.089515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.089525 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:12:52.089555 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.089555 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.089571 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.089676 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:52.090857 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.091683 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.093403 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.093678 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.093789 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.098315 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:52.098317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:52.098317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:52.098326 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:52.100534 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:52.101010 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:52.101011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:52.101012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:52.101080 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:52.101349 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:52.101350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:52.101350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:52.101476 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:52.101557 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.102630 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.102723 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5674,"thread_start_us":59,"threads_started":1}
I20260808 07:12:52.103274 2914 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42779
I20260808 07:12:52.103384 2914 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:52.103602 2914 heartbeater.cc:511] Master 127.22.2.190:42779 requested a full tablet report, sending...
I20260808 07:12:52.104137 2679 ts_manager.cc:194] Registered new tserver with Master: 799dae90dfb7452c9f2b14739997ab60 (127.22.2.129:32881)
I20260808 07:12:52.105023 2679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:60171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:12:52.119495 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:54545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42779
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:52.290112 2922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:52.290364 2922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:52.290408 2922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:52.292303 2922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:52.292375 2922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:52.292424 2922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:12:52.294227 2922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.2.148:54545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:52.294533 2922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:52.294787 2922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:52.309886 2933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:52.310089 2931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:52.309976 2930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:52.310521 2922 server_base.cc:1034] running on GCE node
I20260808 07:12:52.310768 2922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:52.311060 2922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:52.312214 2922 hybrid_clock.cc:648] HybridClock initialized: now 1786173172312200 us; error 32 us; skew 500 ppm
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173172, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:52.316002 2922 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:12:52.316627 2922 webserver.cc:460] Webserver started at http://127.22.2.130:44605/ using document root <none> and password file <none>
I20260808 07:12:52.316910 2922 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:52.316991 2922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:52.317152 2922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173172, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:12:52.341699 2922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "257e1e586d4847c3b194acf3afe64410"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "46a809b289713fcb0455a9a387c54bf6"
server_key_iv: "0c20d61971d825bbc030966d6ab82a48"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.342430 2922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "257e1e586d4847c3b194acf3afe64410"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "46a809b289713fcb0455a9a387c54bf6"
server_key_iv: "0c20d61971d825bbc030966d6ab82a48"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.347508 2922 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173172, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:52.361755 2940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.361989 2922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:12:52.362135 2922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "257e1e586d4847c3b194acf3afe64410"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "46a809b289713fcb0455a9a387c54bf6"
server_key_iv: "0c20d61971d825bbc030966d6ab82a48"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.362270 2922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:52.380777 2922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:52.383641 2922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:52.383831 2922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:52.384328 2922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:52.388808 2922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:52.388880 2922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.388926 2922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:52.388954 2922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.400775 2922 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:37039
I20260808 07:12:52.401290 2922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:12:52.408030 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 2922
I20260808 07:12:52.408213 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:12:52.408488 22538 external_mini_cluster.cc:1428] Setting key 6c822398a35b15e12e7f8389adef61dc
I20260808 07:12:52.415428 3066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:37039 every 8 connection(s)
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173172, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
I20260808 07:12:52.431015 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.415262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:36015 (local address 127.22.2.190:42779)
0808 07:12:52.415309 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.415311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.416017 (+ 706us) server_negotiation.cc:407] Connection header received
0808 07:12:52.416711 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.416714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.416734 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.416835 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:52.418114 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.419041 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.421112 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.421506 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.421584 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.425423 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:52.425425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:52.425425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:52.425434 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:52.427243 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:52.427911 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:52.427913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:52.427913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:52.427981 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:52.428352 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:52.428353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:52.428353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:52.428504 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:52.428572 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.430801 (+ 2229us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.430912 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:12:52.431641 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.414954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42779 (local address 127.22.2.130:36015)
0808 07:12:52.415837 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.415886 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:52.416555 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:52.417033 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:52.417038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:52.417469 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:52.417973 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.417981 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.419185 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.419186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.420989 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.420993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.422180 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.422180 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.422364 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.423190 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:52.423202 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:52.425326 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:52.427339 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:52.427346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:52.427349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:52.427800 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:52.428061 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:52.428062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:52.428062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:52.428213 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:52.428632 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:52.428640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:52.428840 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 07:12:52.431163 (+ 2323us) client_negotiation.cc:240] Negotiation successful
0808 07:12:52.431339 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":91,"threads_started":1}
I20260808 07:12:52.432132 3067 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42779
I20260808 07:12:52.432240 3067 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:52.432447 3067 heartbeater.cc:511] Master 127.22.2.190:42779 requested a full tablet report, sending...
I20260808 07:12:52.432837 2679 ts_manager.cc:194] Registered new tserver with Master: 257e1e586d4847c3b194acf3afe64410 (127.22.2.130:37039)
I20260808 07:12:52.433336 2679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:36015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:12:52.451411 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:54545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42779
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:52.613797 3072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:52.614149 3072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:52.614245 3072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:52.615964 3072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:52.616084 3072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:52.616169 3072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:12:52.618500 3072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.2.148:54545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:12:52.619074 3072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:52.619400 3072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:52.627955 3080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:52.628700 3079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:52.629518 3082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:52.629715 3072 server_base.cc:1034] running on GCE node
I20260808 07:12:52.629926 3072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:52.630201 3072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:52.631367 3072 hybrid_clock.cc:648] HybridClock initialized: now 1786173172631349 us; error 36 us; skew 500 ppm
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173172, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:52.635195 3072 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:12:52.635776 3072 webserver.cc:460] Webserver started at http://127.22.2.131:44383/ using document root <none> and password file <none>
I20260808 07:12:52.636040 3072 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:52.636088 3072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:52.636209 3072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173172, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:12:52.657070 3072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "226b2f1611cc45f69af6356a915808c6"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "410b4d742c33a09eb5eeeedcf14a454b"
server_key_iv: "e85d4c6db9ac2bd3fd07783885a7d656"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.657581 3072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "226b2f1611cc45f69af6356a915808c6"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "410b4d742c33a09eb5eeeedcf14a454b"
server_key_iv: "e85d4c6db9ac2bd3fd07783885a7d656"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.661810 3072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260808 07:12:52.677238 3090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.677415 3072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:12:52.677531 3072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "226b2f1611cc45f69af6356a915808c6"
format_stamp: "Formatted at 2026-08-08 07:12:52 on dist-test-slave-6k22"
server_key: "410b4d742c33a09eb5eeeedcf14a454b"
server_key_iv: "e85d4c6db9ac2bd3fd07783885a7d656"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:52.677649 3072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:52.691053 3072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:52.693533 3072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:52.693708 3072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:52.693948 3072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:52.694346 3072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:52.694399 3072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.694445 3072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:52.694476 3072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:52.709559 3072 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:45063
I20260808 07:12:52.717499 3072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:12:52.719195 3219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:45063 every 8 connection(s)
I20260808 07:12:52.724017 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 3072
I20260808 07:12:52.724193 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:12:52.724412 22538 external_mini_cluster.cc:1428] Setting key 6b21675e06198ab49fc4c4f6db606f61
Aug 08 07:12:52 dist-test-slave-6k22 krb5kdc[2131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173172, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:12:52.741443 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.720831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:55775 (local address 127.22.2.190:42779)
0808 07:12:52.720898 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.720901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.727936 (+ 7035us) server_negotiation.cc:407] Connection header received
0808 07:12:52.728894 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.728895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.728905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.729014 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:52.730245 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.731093 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.732711 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.733016 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.733088 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.737712 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:52.737713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:52.737714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:52.737722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:52.739662 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:52.740190 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:52.740191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:52.740191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:52.740254 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:52.740546 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:52.740547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:52.740547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:52.740698 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:52.740758 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.741252 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.741345 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:12:52.741621 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.720393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42779 (local address 127.22.2.131:55775)
0808 07:12:52.727721 (+ 7328us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.727801 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:52.728766 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:52.729111 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:52.729115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:52.729609 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:52.730122 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.730130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.731219 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.731220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.732599 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.732602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.733174 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.733175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.733344 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.734030 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:52.734041 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:52.737595 (+ 3554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:52.739762 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:52.739782 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:52.739784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:52.740096 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:52.740327 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:52.740328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:52.740328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:52.740461 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:52.740824 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:52.740832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:52.741041 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 07:12:52.741167 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 07:12:52.741344 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7248,"thread_start_us":206,"threads_started":1}
I20260808 07:12:52.742137 3220 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42779
I20260808 07:12:52.742249 3220 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:52.742466 3220 heartbeater.cc:511] Master 127.22.2.190:42779 requested a full tablet report, sending...
I20260808 07:12:52.742846 2679 ts_manager.cc:194] Registered new tserver with Master: 226b2f1611cc45f69af6356a915808c6 (127.22.2.131:45063)
I20260808 07:12:52.743362 2679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:55775
I20260808 07:12:52.749980 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:12:52.769182 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.751830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.22.2.190:42779)
0808 07:12:52.751876 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.751878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.760502 (+ 8624us) server_negotiation.cc:407] Connection header received
0808 07:12:52.760659 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.760660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.760672 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.760716 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:52.761668 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.762429 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.763591 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.763877 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.763928 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.765779 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:52.765780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:52.765781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:52.765789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:52.767641 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:52.768196 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:52.768197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:52.768198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:52.768262 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:52.768493 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:52.768494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:52.768494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:52.768629 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:52.768711 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.768972 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.769061 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:12:52.770961 2679 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:12:52.772934 2679 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:12:52.781356 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.775916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.22.2.129:32881)
0808 07:12:52.776067 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.776072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.776418 (+ 346us) server_negotiation.cc:407] Connection header received
0808 07:12:52.776826 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.776829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.776850 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.776914 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:52.777196 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.778531 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.780058 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.780852 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.780899 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.780961 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.781228 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.781281 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":56,"threads_started":1}
I20260808 07:12:52.781648 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.775541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:32881 (local address 127.0.0.1:50244)
0808 07:12:52.776285 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.776332 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:52.776461 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:52.777031 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:52.777034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:52.777037 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:52.777102 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.777104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.778633 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.778633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.779962 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.779965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.781022 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.781024 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.781106 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.781119 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:12:52.781521 (+ 402us) client_negotiation.cc:240] Negotiation successful
0808 07:12:52.781581 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":609,"threads_started":1}
I20260808 07:12:52.782424 2831 tablet_service.cc:1467] Processing CreateTablet for tablet 87f6573c23334dddaf62ecc89c11cd79 (DEFAULT_TABLE table=test-table [id=5250a5e8035e46ad966a81345d7793ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:52.782884 2831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87f6573c23334dddaf62ecc89c11cd79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:52.786829 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.776279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:45063 (local address 127.0.0.1:59942)
0808 07:12:52.782012 (+ 5733us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.782022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:52.782063 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:52.782600 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:52.782600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:52.782602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:52.782632 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.782633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.784151 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.784152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.785335 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.785337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.786612 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.786614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.786671 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.786674 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:12:52.786704 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 07:12:52.786742 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5720,"thread_start_us":111,"threads_started":1}
I20260808 07:12:52.787384 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.776587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.22.2.131:45063)
0808 07:12:52.782250 (+ 5663us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.782254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.782265 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:12:52.782320 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.782322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.782336 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.782496 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:52.782717 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.784046 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.785536 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.786473 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.787027 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.787155 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.787183 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.787222 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5616,"thread_start_us":105,"threads_started":1}
I20260808 07:12:52.788163 3148 tablet_service.cc:1467] Processing CreateTablet for tablet 87f6573c23334dddaf62ecc89c11cd79 (DEFAULT_TABLE table=test-table [id=5250a5e8035e46ad966a81345d7793ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:52.788545 3148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87f6573c23334dddaf62ecc89c11cd79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:52.790279 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.775853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.22.2.130:37039)
0808 07:12:52.776192 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.776196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.776968 (+ 772us) server_negotiation.cc:407] Connection header received
0808 07:12:52.777107 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.777109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.777125 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.777179 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:52.777531 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.778996 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.780345 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.787116 (+ 6771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.787168 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.787347 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.790047 (+ 2700us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.790095 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":106,"threads_started":1}
I20260808 07:12:52.790808 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.775665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:37039 (local address 127.0.0.1:59538)
0808 07:12:52.776899 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.776909 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:52.776998 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:52.777313 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:52.777314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:52.777316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:52.777441 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.777442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.779089 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.779090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.780156 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.780158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.789762 (+ 9604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.789763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.789933 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.789937 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:12:52.790635 (+ 698us) client_negotiation.cc:240] Negotiation successful
0808 07:12:52.790688 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1200,"thread_start_us":131,"threads_started":1}
I20260808 07:12:52.791767 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 87f6573c23334dddaf62ecc89c11cd79 (DEFAULT_TABLE table=test-table [id=5250a5e8035e46ad966a81345d7793ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:52.792164 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87f6573c23334dddaf62ecc89c11cd79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:52.793220 3240 tablet_bootstrap.cc:492] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60: Bootstrap starting.
I20260808 07:12:52.793249 3243 tablet_bootstrap.cc:492] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6: Bootstrap starting.
I20260808 07:12:52.794317 3240 tablet_bootstrap.cc:654] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:52.794771 3240 log.cc:824] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:52.795506 3243 tablet_bootstrap.cc:654] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:52.796286 3243 log.cc:824] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:52.797185 3240 tablet_bootstrap.cc:492] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60: No bootstrap required, opened a new log
I20260808 07:12:52.797201 3243 tablet_bootstrap.cc:492] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6: No bootstrap required, opened a new log
I20260808 07:12:52.797264 3240 ts_tablet_manager.cc:1397] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 07:12:52.797264 3243 ts_tablet_manager.cc:1397] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:12:52.801800 3243 raft_consensus.cc:348] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.801967 3243 raft_consensus.cc:374] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:52.802003 3243 raft_consensus.cc:729] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226b2f1611cc45f69af6356a915808c6, State: Initialized, Role: FOLLOWER
I20260808 07:12:52.802323 3243 consensus_queue.cc:260] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.802446 3240 raft_consensus.cc:348] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.802606 3240 raft_consensus.cc:374] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:52.802675 3240 raft_consensus.cc:729] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799dae90dfb7452c9f2b14739997ab60, State: Initialized, Role: FOLLOWER
I20260808 07:12:52.802837 3240 consensus_queue.cc:260] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.802992 3243 ts_tablet_manager.cc:1428] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 07:12:52.803176 3220 heartbeater.cc:503] Master 127.22.2.190:42779 was elected leader, sending a full tablet report...
I20260808 07:12:52.803403 2914 heartbeater.cc:503] Master 127.22.2.190:42779 was elected leader, sending a full tablet report...
I20260808 07:12:52.803133 3240 ts_tablet_manager.cc:1428] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 07:12:52.805234 3246 tablet_bootstrap.cc:492] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410: Bootstrap starting.
I20260808 07:12:52.806720 3246 tablet_bootstrap.cc:654] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:52.807523 3246 log.cc:824] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:52.808394 3246 tablet_bootstrap.cc:492] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410: No bootstrap required, opened a new log
I20260808 07:12:52.808482 3246 ts_tablet_manager.cc:1397] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:12:52.810946 3246 raft_consensus.cc:348] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.811100 3246 raft_consensus.cc:374] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:52.811156 3246 raft_consensus.cc:729] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257e1e586d4847c3b194acf3afe64410, State: Initialized, Role: FOLLOWER
I20260808 07:12:52.811311 3246 consensus_queue.cc:260] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.811745 3246 ts_tablet_manager.cc:1428] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:12:52.811882 3067 heartbeater.cc:503] Master 127.22.2.190:42779 was elected leader, sending a full tablet report...
I20260808 07:12:52.823258 3248 raft_consensus.cc:482] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:12:52.823411 3248 raft_consensus.cc:504] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.823913 3248 leader_election.cc:288] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 257e1e586d4847c3b194acf3afe64410 (127.22.2.130:37039), 226b2f1611cc45f69af6356a915808c6 (127.22.2.131:45063)
I20260808 07:12:52.828928 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.824676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:48565 (local address 127.22.2.130:37039)
0808 07:12:52.824729 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.824730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.824739 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:12:52.824763 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.824763 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.824773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.824829 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:52.825111 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.825978 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.827931 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.828469 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.828517 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.828525 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.828818 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.828858 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:12:52.828928 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.824393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:33853 (local address 127.22.2.131:45063)
0808 07:12:52.824487 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.824489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.825000 (+ 511us) server_negotiation.cc:407] Connection header received
0808 07:12:52.825112 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.825113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.825126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.825159 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:52.825461 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.826187 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.827931 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.828469 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.828517 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.828525 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.828818 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.828858 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 07:12:52.829227 3253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.824069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:45063 (local address 127.22.2.129:33853)
0808 07:12:52.824919 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.824929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:52.825031 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:52.825248 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:52.825248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:52.825250 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:52.825373 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.825374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.826404 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.826405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.827823 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.827826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.828589 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.828590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.828728 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.828730 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:12:52.829045 (+ 315us) client_negotiation.cc:240] Negotiation successful
0808 07:12:52.829102 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":74,"thread_start_us":536,"threads_started":1}
I20260808 07:12:52.829599 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.824050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:37039 (local address 127.22.2.129:48565)
0808 07:12:52.824621 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.824633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:52.824698 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:52.824834 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:52.824835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:52.824849 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:52.825024 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.825027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.826150 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.826151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.827823 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:52.827826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.828590 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.828591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:52.828720 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.828725 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:12:52.829446 (+ 721us) client_negotiation.cc:240] Negotiation successful
0808 07:12:52.829494 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":41,"thread_start_us":381,"threads_started":1}
I20260808 07:12:52.829629 3168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87f6573c23334dddaf62ecc89c11cd79" candidate_uuid: "799dae90dfb7452c9f2b14739997ab60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "226b2f1611cc45f69af6356a915808c6" is_pre_election: true
I20260808 07:12:52.829794 3168 raft_consensus.cc:2454] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 799dae90dfb7452c9f2b14739997ab60 in term 0.
I20260808 07:12:52.830037 2787 leader_election.cc:302] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 226b2f1611cc45f69af6356a915808c6, 799dae90dfb7452c9f2b14739997ab60; no voters:
I20260808 07:12:52.830170 3248 raft_consensus.cc:2781] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:12:52.830279 3248 raft_consensus.cc:482] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:12:52.830312 3248 raft_consensus.cc:3037] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:52.830289 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87f6573c23334dddaf62ecc89c11cd79" candidate_uuid: "799dae90dfb7452c9f2b14739997ab60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257e1e586d4847c3b194acf3afe64410" is_pre_election: true
I20260808 07:12:52.830518 3018 raft_consensus.cc:2454] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 799dae90dfb7452c9f2b14739997ab60 in term 0.
I20260808 07:12:52.831467 3248 raft_consensus.cc:504] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.831646 3248 leader_election.cc:288] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [CANDIDATE]: Term 1 election: Requested vote from peers 257e1e586d4847c3b194acf3afe64410 (127.22.2.130:37039), 226b2f1611cc45f69af6356a915808c6 (127.22.2.131:45063)
I20260808 07:12:52.831852 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87f6573c23334dddaf62ecc89c11cd79" candidate_uuid: "799dae90dfb7452c9f2b14739997ab60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257e1e586d4847c3b194acf3afe64410"
I20260808 07:12:52.831911 3168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87f6573c23334dddaf62ecc89c11cd79" candidate_uuid: "799dae90dfb7452c9f2b14739997ab60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "226b2f1611cc45f69af6356a915808c6"
I20260808 07:12:52.831943 3018 raft_consensus.cc:3037] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:52.831977 3168 raft_consensus.cc:3037] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6 [term 0 FOLLOWER]: Advancing to term 1
W20260808 07:12:52.833456 2915 tablet.cc:2369] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:12:52.833534 3168 raft_consensus.cc:2454] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 799dae90dfb7452c9f2b14739997ab60 in term 1.
I20260808 07:12:52.833534 3018 raft_consensus.cc:2454] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 799dae90dfb7452c9f2b14739997ab60 in term 1.
I20260808 07:12:52.833741 2786 leader_election.cc:302] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257e1e586d4847c3b194acf3afe64410, 799dae90dfb7452c9f2b14739997ab60; no voters:
I20260808 07:12:52.833839 3248 raft_consensus.cc:2781] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:12:52.834004 3248 raft_consensus.cc:686] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [term 1 LEADER]: Becoming Leader. State: Replica: 799dae90dfb7452c9f2b14739997ab60, State: Running, Role: LEADER
I20260808 07:12:52.834118 3248 consensus_queue.cc:237] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } }
I20260808 07:12:52.835641 2679 catalog_manager.cc:5495] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 799dae90dfb7452c9f2b14739997ab60 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "799dae90dfb7452c9f2b14739997ab60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799dae90dfb7452c9f2b14739997ab60" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 } health_report { overall_health: UNKNOWN } } }
I20260808 07:12:52.866639 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.860071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50260 (local address 127.22.2.129:32881)
0808 07:12:52.860145 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:52.860148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:52.863300 (+ 3152us) server_negotiation.cc:407] Connection header received
0808 07:12:52.863345 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:52.863346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:52.863372 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:52.863441 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:12:52.863709 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.864439 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.865494 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:52.865793 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:52.865855 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:52.866048 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:12:52.866180 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:12:52.866263 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:52.866493 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 07:12:52.866512 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:12:52.869068 3018 raft_consensus.cc:1261] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 1 FOLLOWER]: Refusing update from remote peer 799dae90dfb7452c9f2b14739997ab60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:12:52.869318 3168 raft_consensus.cc:1261] T 87f6573c23334dddaf62ecc89c11cd79 P 226b2f1611cc45f69af6356a915808c6 [term 1 FOLLOWER]: Refusing update from remote peer 799dae90dfb7452c9f2b14739997ab60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:12:52.869474 3248 consensus_queue.cc:1035] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "257e1e586d4847c3b194acf3afe64410" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 37039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:12:52.869846 3248 consensus_queue.cc:1035] T 87f6573c23334dddaf62ecc89c11cd79 P 799dae90dfb7452c9f2b14739997ab60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "226b2f1611cc45f69af6356a915808c6" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:12:52.876669 3257 mvcc.cc:205] Tried to move back new op lower bound from 7316165316068864000 to 7316165315929120768. Current Snapshot: MvccSnapshot[applied={T|T < 7316165316068864000}]
I20260808 07:12:52.876926 3258 mvcc.cc:205] Tried to move back new op lower bound from 7316165316068864000 to 7316165315929120768. Current Snapshot: MvccSnapshot[applied={T|T < 7316165316068864000}]
I20260808 07:12:52.879766 3260 mvcc.cc:205] Tried to move back new op lower bound from 7316165316068864000 to 7316165315929120768. Current Snapshot: MvccSnapshot[applied={T|T < 7316165316068864000}]
I20260808 07:12:52.883060 2679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:12:52.883181 2679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:12:52.884093 2679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8bf34c3b28c440f85b7919fa549588c: Sending DeleteTablet for 3 replicas of tablet 87f6573c23334dddaf62ecc89c11cd79
I20260808 07:12:52.884892 2998 tablet_service.cc:1514] Processing DeleteTablet for tablet 87f6573c23334dddaf62ecc89c11cd79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:12:52 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:59538
I20260808 07:12:52.885367 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 2766
I20260808 07:12:52.889246 3148 tablet_service.cc:1514] Processing DeleteTablet for tablet 87f6573c23334dddaf62ecc89c11cd79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:12:52 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:59942
I20260808 07:12:52.895485 3268 tablet_replica.cc:330] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410: stopping tablet replica
W20260808 07:12:52.895651 2665 connection.cc:535] client connection to 127.22.2.129:32881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:12:52.895684 3268 raft_consensus.cc:2229] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 07:12:52.895771 2665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:12:52.895938 3268 raft_consensus.cc:2258] T 87f6573c23334dddaf62ecc89c11cd79 P 257e1e586d4847c3b194acf3afe64410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:12:52.896334 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 2922
I20260808 07:12:52.905373 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 3072
I20260808 07:12:52.907049 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.906772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:32881 (local address 127.0.0.1:50264)
0808 07:12:52.906965 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.907007 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260808 07:12:52.907145 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:52.906915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:37039 (local address 127.0.0.1:59552)
0808 07:12:52.907114 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:52.907132 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.130:37039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
W20260808 07:12:52.907248 2664 catalog_manager.cc:4584] TS 257e1e586d4847c3b194acf3afe64410 (127.22.2.130:37039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87f6573c23334dddaf62ecc89c11cd79: Network error: Client connection negotiation failed: client connection to 127.22.2.130:37039: connect: Connection refused (error 111)
I20260808 07:12:52.913643 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 2641
I20260808 07:12:52.923261 22538 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:12:53.267982 22538 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:12:53.268093 22538 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:12:53.612671 22538 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:12:53.612 UTC [2165] LOG: received smart shutdown request
2026-08-08 07:12:53.616 UTC [2165] LOG: background worker "logical replication launcher" (PID 2175) exited with exit code 1
2026-08-08 07:12:53.617 UTC [2170] LOG: shutting down
2026-08-08 07:12:53.797 UTC [2165] LOG: database system is shut down
2026-08-08T07:12:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47040 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:12:53 dist-test-slave-6k22 krb5kdc[3282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:12:53 dist-test-slave-6k22 krb5kdc[3282](info): set up 2 sockets
Aug 08 07:12:53 dist-test-slave-6k22 krb5kdc[3282](info): commencing operation
krb5kdc: starting...
W20260808 07:12:55.018090 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:12:55 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:12:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/rangeradmin_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/rangeradmin_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/rangerlookup_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/rangerlookup_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
I20260808 07:12:55.185732 22538 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskyv8zyi/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/postgres -l logfile start
2026-08-08 07:12:57.771 UTC [3317] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:12:57.771 UTC [3317] LOG: listening on IPv4 address "127.22.2.148", port 39431
2026-08-08 07:12:57.775 UTC [3317] LOG: listening on Unix socket "/tmp/.s.PGSQL.39431"
2026-08-08 07:12:57.797 UTC [3320] LOG: database system was shut down at 2026-08-08 07:12:56 UTC
2026-08-08 07:12:57.801 UTC [3317] LOG: database system is ready to accept connections
I20260808 07:12:58.841465 22538 mini_postgres.cc:96] Postgres bound to 39431
2026-08-08 07:12:58.848 UTC [3330] FATAL: database "slave" does not exist
127.22.2.148:39431 - accepting connections
I20260808 07:12:58.849270 22538 mini_ranger.cc:161] Starting Ranger...
I20260808 07:12:58.865372 22538 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:12:59.364394 22538 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:12:59.364470 22538 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-admin
2026-08-08 07:12:59,652 [I] DB FLAVOR :POSTGRES
2026-08-08 07:12:59,652 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:12:59,653 [I] Checking connection..
2026-08-08 07:12:59,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:12:59,930 [I] Checking connection passed.
2026-08-08 07:12:59,930 [I] --------- Verifying version history table ---------
2026-08-08 07:12:59,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:13:00,191 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:13:00,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:13:00,427 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:13:00,427 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:13:00,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:13:00.685 UTC [3400] WARNING: there is no transaction in progress
2026-08-08 07:13:00,693 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:13:00,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:13:00,959 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:13:00,959 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:13:00,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:13:01,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:13:01,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-08 07:13:01,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:13:02,010 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:13:02,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:13:02,259 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:13:02,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:13:02,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:13:02,776 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:13:02,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:13:03.740 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.757 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.774 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.790 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.807 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.827 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.881 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.888 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.902 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.913 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.924 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.940 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.950 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.957 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.965 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:03.972 UTC [3519] WARNING: there is no transaction in progress
2026-08-08 07:13:04,644 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:13:04,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:13:04,891 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:13:04,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:13:05,143 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:13:05,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:13:05,405 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:13:05,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:13:05,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:13:05,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-08 07:13:06,139 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:13:06,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:13:06,391 [I] DB_PATCHES have already been applied
I20260808 07:13:06.397593 22538 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-admin:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskyv8zyi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:13:06.397673 22538 mini_ranger.cc:193] Using host: 127.22.2.148
I20260808 07:13:06.401319 22538 mini_ranger.cc:227] Ranger admin URL: http://127.22.2.148:60967
Aug 08, 2026 7:13:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:13:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:13:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:13:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:13:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:13:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:13:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:13:07.599917 22538 mini_ranger.cc:160] Time spent starting Ranger: real 8.751s user 0.000s sys 0.010s
Aug 08, 2026 7:13:28 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:13:28 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 7:13:28 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:13:28 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:13:29 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:13:29.943353 22538 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.2.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.2.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/rangerkms_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.2.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/rangerkms_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.2.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.2.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.2.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/HTTP_127.22.2.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:13:30 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173210, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:13:30.033646 22538 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:13:30.043313 22538 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:13:30.897714 22538 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:13:30.897816 22538 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms
I20260808 07:13:30.897838 22538 mini_ranger_kms.cc:227] Using postgres at 127.22.2.148:39431
2026-08-08 07:13:31,062 [I] DB FLAVOR :POSTGRES
2026-08-08 07:13:31,062 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:13:31,062 [I] Checking connection
2026-08-08 07:13:31,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:13:31,305 [I] connection success
2026-08-08 07:13:31,305 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:13:31,305 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:13:31,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:13:31,536 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:13:31,536 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:13:31,537 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:13:31,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.2.148:39431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:13:31,830 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:13:32.312274 22538 mini_ranger_kms.cc:310] Created kms service
I20260808 07:13:32.498819 22538 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:13:32.542204 22538 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:13:32 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173210, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:32.971977 22538 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:13:32.972196 22538 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms:/tmp/dist-test-taskyv8zyi/build/release/bin/postgresql.jar:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskyv8zyi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:13:32.972227 22538 mini_ranger_kms.cc:240] Using host: 127.22.2.148
I20260808 07:13:32.976246 22538 mini_ranger_kms.cc:276] Ranger KMS PID: 3720
I20260808 07:13:32.976331 22538 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.2.148:51429
Aug 08, 2026 7:13:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:13:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:13:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:13:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:13:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:13:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 07:13:35.335376 22538 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.302s user 0.002s sys 0.147s
I20260808 07:13:35.335484 22538 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:13:35.335531 22538 mini_ranger_kms.cc:401] 127.22.2.148:51429/kms/v1/keys
Aug 08 07:13:35 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173215, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 07:13:35 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173215, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
Aug 08 07:13:36 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173215, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.2.148@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
Aug 08 07:13:38 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:13:38.395129 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46817
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:46817
--ranger_config_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:51429/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:38.479609 3776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:38.479863 3776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:38.479900 3776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:38.481149 3776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:38.481215 3776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:38.481236 3776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:38.481258 3776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:38.481300 3776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:38.482847 3776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.2.148:51429/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:46817
--ranger_config_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:38.483105 3776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:38.483333 3776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:13:38.489705 3776 server_base.cc:1034] running on GCE node
W20260808 07:13:38.489672 3783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:38.489804 3785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:38.489653 3782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:38.490305 3776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:38.490624 3776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:38.491796 3776 hybrid_clock.cc:648] HybridClock initialized: now 1786173218491792 us; error 49 us; skew 500 ppm
Aug 08 07:13:38 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173218, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:38.494817 3776 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:13:38.495296 3776 webserver.cc:460] Webserver started at http://127.22.2.190:46001/ using document root <none> and password file <none>
I20260808 07:13:38.495543 3776 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:38.495652 3776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:38.495817 3776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:13:38 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173218, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:38.546990 3776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "8b5fe885a6564a2b82e898a645939289"
format_stamp: "Formatted at 2026-08-08 07:13:38 on dist-test-slave-6k22"
server_key: "2cee7f631ba3a438418956f34fa7de8d"
server_key_iv: "5055287ecad6fa1e3faca43482427629"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:38.547387 3776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "8b5fe885a6564a2b82e898a645939289"
format_stamp: "Formatted at 2026-08-08 07:13:38 on dist-test-slave-6k22"
server_key: "2cee7f631ba3a438418956f34fa7de8d"
server_key_iv: "5055287ecad6fa1e3faca43482427629"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:38.548926 3776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:38.560434 3793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:38.560669 3776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:38.560827 3776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "8b5fe885a6564a2b82e898a645939289"
format_stamp: "Formatted at 2026-08-08 07:13:38 on dist-test-slave-6k22"
server_key: "2cee7f631ba3a438418956f34fa7de8d"
server_key_iv: "5055287ecad6fa1e3faca43482427629"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:38.560971 3776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:38.591017 3776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:38.592892 3776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:38.593065 3776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:38.597493 3776 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:46817
I20260808 07:13:38.597546 3853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:46817 every 8 connection(s)
I20260808 07:13:38.597895 3776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:38.598573 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:38.601071 3854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Bootstrap starting.
I20260808 07:13:38.601751 3854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:38.602080 3854 log.cc:824] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:38.602479 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 3776
I20260808 07:13:38.602641 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:13:38.602846 22538 external_mini_cluster.cc:1428] Setting key 06c4554931898e126ba37cd9658df4a7
I20260808 07:13:38.602929 3854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: No bootstrap required, opened a new log
I20260808 07:13:38.605530 3854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } }
I20260808 07:13:38.605693 3854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:38.605720 3854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5fe885a6564a2b82e898a645939289, State: Initialized, Role: FOLLOWER
I20260808 07:13:38.605875 3854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } }
I20260808 07:13:38.605948 3854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:38.605985 3854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:38.606031 3854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:38.606645 3854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } }
I20260808 07:13:38.606804 3854 leader_election.cc:302] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b5fe885a6564a2b82e898a645939289; no voters:
I20260808 07:13:38.607028 3854 leader_election.cc:288] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:13:38.607056 3860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:38.607255 3860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5fe885a6564a2b82e898a645939289, State: Running, Role: LEADER
I20260808 07:13:38.607381 3854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:38.607476 3860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } }
I20260808 07:13:38.607867 3860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b5fe885a6564a2b82e898a645939289. Latest consensus state: current_term: 1 leader_uuid: "8b5fe885a6564a2b82e898a645939289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } } }
I20260808 07:13:38.607968 3860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:38.608180 3867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:38.608749 3867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:38.609126 3861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b5fe885a6564a2b82e898a645939289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } } }
I20260808 07:13:38.609179 3854 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:13:38.609290 3861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: This master's current role is: LEADER
Aug 08 07:13:38 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:38.614722 3867 catalog_manager.cc:1347] Generated new cluster ID: be41f7460d5c485a89cce4848e301edb
I20260808 07:13:38.614853 3867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:38.617939 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:38.603415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.22.2.190:46817)
0808 07:13:38.603642 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:38.603651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:38.603685 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:13:38.604253 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:38.604259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:38.604304 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:38.604459 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:38.605080 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:38.606201 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:38.607063 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:38.607669 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:38.607780 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:38.612098 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:38.612167 (+ 69us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:38.612168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:38.612250 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:38.615650 (+ 3400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:38.616126 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:38.616129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:38.616129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:38.616253 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:38.616500 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:38.616501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:38.616502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:38.616849 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:38.617439 (+ 590us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:38.617518 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:13:38.617694 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260808 07:13:38.658679 3867 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:38.659260 3867 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:38.664336 3867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Generated new TSK 0
I20260808 07:13:38.664538 3867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:13:39.387701 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:51429/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46817
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:39.497817 3904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:39.498159 3904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:39.498235 3904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:39.499974 3904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:39.500027 3904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:39.500066 3904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:13:39.502149 3904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.2.148:51429/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:39.502395 3904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:39.502681 3904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:39.509200 3914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:39.509275 3904 server_base.cc:1034] running on GCE node
W20260808 07:13:39.509358 3910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:39.509373 3912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:39.509642 3904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:39.509928 3904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:39.511107 3904 hybrid_clock.cc:648] HybridClock initialized: now 1786173219511095 us; error 31 us; skew 500 ppm
Aug 08 07:13:39 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173219, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:39.514142 3904 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:13:39.514573 3904 webserver.cc:460] Webserver started at http://127.22.2.129:41721/ using document root <none> and password file <none>
I20260808 07:13:39.514814 3904 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:39.514894 3904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:39.515044 3904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:13:39 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173219, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:39.535120 3904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "3ec85f1d70b64732830fbf530481810f"
format_stamp: "Formatted at 2026-08-08 07:13:39 on dist-test-slave-6k22"
server_key: "9350780e4366b28ddcc2a80e1f0960b9"
server_key_iv: "b7a4cc5889f23461818afa34fcf68d23"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:39.535681 3904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "3ec85f1d70b64732830fbf530481810f"
format_stamp: "Formatted at 2026-08-08 07:13:39 on dist-test-slave-6k22"
server_key: "9350780e4366b28ddcc2a80e1f0960b9"
server_key_iv: "b7a4cc5889f23461818afa34fcf68d23"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:39.539193 3904 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 07:13:39.559728 3923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:39.559968 3904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 07:13:39.560061 3904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "3ec85f1d70b64732830fbf530481810f"
format_stamp: "Formatted at 2026-08-08 07:13:39 on dist-test-slave-6k22"
server_key: "9350780e4366b28ddcc2a80e1f0960b9"
server_key_iv: "b7a4cc5889f23461818afa34fcf68d23"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:39.560150 3904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:39.571317 3904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:39.573709 3904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:39.573899 3904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:39.574121 3904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:39.574507 3904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:39.574584 3904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:39.574648 3904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:39.574682 3904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:39.581691 3904 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33849
I20260808 07:13:39.581895 4061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33849 every 8 connection(s)
I20260808 07:13:39.582165 3904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:39.585284 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 3904
I20260808 07:13:39.585391 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:13:39.585575 22538 external_mini_cluster.cc:1428] Setting key b97a5224694c98a7f6e8822435234a93
Aug 08 07:13:39 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173219, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:39.593832 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:39.582987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:58077 (local address 127.22.2.190:46817)
0808 07:13:39.583131 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:39.583134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:39.583352 (+ 218us) server_negotiation.cc:407] Connection header received
0808 07:13:39.584104 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:39.584106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:39.584120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:39.584184 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:39.584976 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:39.585785 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:39.587340 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:39.587689 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:39.587775 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:39.590665 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:39.590666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:39.590667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:39.590675 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:39.592365 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:39.592797 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:39.592798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:39.592799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:39.592877 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:39.593150 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:39.593151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:39.593151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:39.593264 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:39.593335 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:39.593609 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 07:13:39.593696 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260808 07:13:39.594002 4064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:39.582947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46817 (local address 127.22.2.129:58077)
0808 07:13:39.583206 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:39.583252 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:39.583958 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:39.584274 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:39.584279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:39.584552 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:39.584859 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:39.584866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:39.585962 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:39.585963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:39.587231 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:39.587234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:39.587867 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:39.587868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:39.588019 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:39.588823 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:39.588833 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:39.590554 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:39.592468 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:39.592475 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:39.592478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:39.592703 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:39.592956 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:39.592957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:39.592957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:39.593076 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:39.593329 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:39.593336 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:39.593472 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 07:13:39.593574 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:13:39.593716 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260808 07:13:39.594542 4062 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46817
I20260808 07:13:39.594651 4062 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:39.594861 4062 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:39.595386 3812 ts_manager.cc:194] Registered new tserver with Master: 3ec85f1d70b64732830fbf530481810f (127.22.2.129:33849)
I20260808 07:13:39.596068 3812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:58077
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:13:39.622733 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:51429/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46817
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:39.723069 4068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:39.723284 4068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:39.723376 4068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:39.724577 4068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:39.724642 4068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:39.724686 4068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
Aug 08 07:13:39 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173219, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:39.726401 4068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.2.148:51429/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:39.726660 4068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:39.726887 4068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:39.732321 4078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:39.732348 4076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:39.732563 4068 server_base.cc:1034] running on GCE node
W20260808 07:13:39.732777 4075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:39.733026 4068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:39.733301 4068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:39.734472 4068 hybrid_clock.cc:648] HybridClock initialized: now 1786173219734435 us; error 51 us; skew 500 ppm
Aug 08 07:13:39 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173219, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:39.737455 4068 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:13:39.737928 4068 webserver.cc:460] Webserver started at http://127.22.2.130:34287/ using document root <none> and password file <none>
I20260808 07:13:39.738191 4068 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:39.738243 4068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:39.738394 4068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:13:39 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173219, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:39.755849 4068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "6387d2795d6f4c54b1e96a5272e7a29d"
format_stamp: "Formatted at 2026-08-08 07:13:39 on dist-test-slave-6k22"
server_key: "e21fd2093aaebf646da17ffdd4504452"
server_key_iv: "7679c6ba8b45239830061b5afd1f4221"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:39.756415 4068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "6387d2795d6f4c54b1e96a5272e7a29d"
format_stamp: "Formatted at 2026-08-08 07:13:39 on dist-test-slave-6k22"
server_key: "e21fd2093aaebf646da17ffdd4504452"
server_key_iv: "7679c6ba8b45239830061b5afd1f4221"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:39.760515 4068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 07:13:39.779714 4086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:39.779958 4068 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 07:13:39.780094 4068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "6387d2795d6f4c54b1e96a5272e7a29d"
format_stamp: "Formatted at 2026-08-08 07:13:39 on dist-test-slave-6k22"
server_key: "e21fd2093aaebf646da17ffdd4504452"
server_key_iv: "7679c6ba8b45239830061b5afd1f4221"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:39.780220 4068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:39.805539 4068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:39.808385 4068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:39.808593 4068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:39.808928 4068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:39.809402 4068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:39.809495 4068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:39.809568 4068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:39.809628 4068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:39.818049 4068 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:45127
I20260808 07:13:39.818562 4068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:13:39.819131 4215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:45127 every 8 connection(s)
I20260808 07:13:39.826503 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 4068
I20260808 07:13:39.826642 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:13:39.826855 22538 external_mini_cluster.cc:1428] Setting key c835f8231084954e478b55d7fe7a6e78
Aug 08 07:13:39 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173219, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:39.831457 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:39.820451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:48639 (local address 127.22.2.190:46817)
0808 07:13:39.820502 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:39.820505 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:39.820760 (+ 255us) server_negotiation.cc:407] Connection header received
0808 07:13:39.821512 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:39.821513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:39.821527 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:39.821558 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:39.822405 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:39.823216 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:39.824756 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:39.825079 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:39.825126 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:39.828080 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:39.828081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:39.828082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:39.828089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:39.829869 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:39.830350 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:39.830351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:39.830351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:39.830410 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:39.830680 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:39.830681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:39.830681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:39.830822 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:39.830871 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:39.831263 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 07:13:39.831358 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:13:39.832257 4218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:39.820167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46817 (local address 127.22.2.130:48639)
0808 07:13:39.820629 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:39.820674 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:39.821383 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:39.821745 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:39.821750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:39.822009 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:39.822288 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:39.822294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:39.823336 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:39.823337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:39.824653 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:39.824656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:39.825233 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:39.825234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:39.825384 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:39.826007 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:39.826018 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:39.827967 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:39.829971 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:39.829977 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:39.829979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:39.830258 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:39.830481 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:39.830481 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:39.830482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:39.830607 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:39.830933 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:39.830940 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:39.831096 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 07:13:39.831858 (+ 762us) client_negotiation.cc:240] Negotiation successful
0808 07:13:39.832020 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":81,"threads_started":1}
I20260808 07:13:39.832906 4216 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46817
I20260808 07:13:39.832996 4216 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:39.833173 4216 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:39.833564 3812 ts_manager.cc:194] Registered new tserver with Master: 6387d2795d6f4c54b1e96a5272e7a29d (127.22.2.130:45127)
I20260808 07:13:39.834044 3812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:48639
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:13:39.867393 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:51429/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46817
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:40.006855 4221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:40.007289 4221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:40.007408 4221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:40.009395 4221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:40.009536 4221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:40.009670 4221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:13:40.012180 4221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.2.148:51429/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:40.012739 4221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:40.013096 4221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:40.020113 4231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.021337 4228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.021508 4227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:40.028956 4221 server_base.cc:1034] running on GCE node
I20260808 07:13:40.029250 4221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:40.029603 4221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:40.037014 4221 hybrid_clock.cc:648] HybridClock initialized: now 1786173220036978 us; error 49 us; skew 500 ppm
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:40.040831 4221 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:13:40.041447 4221 webserver.cc:460] Webserver started at http://127.22.2.131:38007/ using document root <none> and password file <none>
I20260808 07:13:40.041734 4221 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:40.041802 4221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:40.041998 4221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:40.059703 4221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "9b11d0150f424495b5dc2e0872821841"
format_stamp: "Formatted at 2026-08-08 07:13:40 on dist-test-slave-6k22"
server_key: "5c567925160ef2d75dc7b8e5cb9c1834"
server_key_iv: "1779cbe0e25eb2ae2dede496e850457a"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:40.060348 4221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "9b11d0150f424495b5dc2e0872821841"
format_stamp: "Formatted at 2026-08-08 07:13:40 on dist-test-slave-6k22"
server_key: "5c567925160ef2d75dc7b8e5cb9c1834"
server_key_iv: "1779cbe0e25eb2ae2dede496e850457a"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:40.063202 4221 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 07:13:40.093433 4240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.093673 4221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:40.093760 4221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "9b11d0150f424495b5dc2e0872821841"
format_stamp: "Formatted at 2026-08-08 07:13:40 on dist-test-slave-6k22"
server_key: "5c567925160ef2d75dc7b8e5cb9c1834"
server_key_iv: "1779cbe0e25eb2ae2dede496e850457a"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:40.093848 4221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:40.111039 4221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:40.113626 4221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:40.113821 4221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:40.114082 4221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:40.114460 4221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:40.114542 4221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.114612 4221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:40.114670 4221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.121755 4221 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:39473
I20260808 07:13:40.122238 4221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:13:40.123680 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 4221
I20260808 07:13:40.123793 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:13:40.123996 22538 external_mini_cluster.cc:1428] Setting key 767c530f3c24d8fd77ed92cfe1b6321e
I20260808 07:13:40.126832 4369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:39473 every 8 connection(s)
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:40.138259 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:40.126034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:42051 (local address 127.22.2.190:46817)
0808 07:13:40.126077 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:40.126079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:40.127308 (+ 1229us) server_negotiation.cc:407] Connection header received
0808 07:13:40.128288 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:40.128289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:40.128302 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:40.128333 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:40.129152 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.129886 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.131327 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.131648 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.131693 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:40.134291 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:40.134292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:40.134293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:40.134300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:40.136681 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:40.137253 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:40.137254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:40.137254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:40.137311 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:40.137567 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:40.137568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:40.137568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:40.137687 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:40.137727 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:40.138076 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 07:13:40.138172 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:13:40.139017 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:40.125768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46817 (local address 127.22.2.131:42051)
0808 07:13:40.127168 (+ 1400us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:40.127218 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:40.127997 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:40.128517 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:40.128522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:40.128768 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:40.129033 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:40.129038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.129998 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.129999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:40.131227 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:40.131230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.131796 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.131797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:40.131921 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:40.132514 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:40.132525 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:40.134183 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:40.136787 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:40.136793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:40.136795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:40.137146 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:40.137378 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:40.137379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:40.137379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:40.137499 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:40.137771 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:40.137777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:40.137937 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:13:40.138651 (+ 714us) client_negotiation.cc:240] Negotiation successful
0808 07:13:40.138806 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1344,"thread_start_us":85,"threads_started":1}
I20260808 07:13:40.139829 4370 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46817
I20260808 07:13:40.139914 4370 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:40.140079 4370 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:40.140463 3812 ts_manager.cc:194] Registered new tserver with Master: 9b11d0150f424495b5dc2e0872821841 (127.22.2.131:39473)
I20260808 07:13:40.140899 3812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:42051
I20260808 07:13:40.146912 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:40.147749 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 3904
I20260808 07:13:40.200430 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 4068
I20260808 07:13:40.207755 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 4221
I20260808 07:13:40.214710 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 3776
I20260808 07:13:40.225179 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46817
--webserver_interface=127.22.2.190
--webserver_port=46001
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:46817
--ranger_config_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:51429/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:51429/kms with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:40.308264 4374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:40.308503 4374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:40.308543 4374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:40.309689 4374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:40.309754 4374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:40.309782 4374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:40.309811 4374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:40.309835 4374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:40.311443 4374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:51429/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:46817
--ranger_config_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:40.311738 4374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:40.311969 4374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:40.317042 4381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.317112 4382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.317162 4384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:40.317355 4374 server_base.cc:1034] running on GCE node
I20260808 07:13:40.317660 4374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:40.317968 4374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:40.319154 4374 hybrid_clock.cc:648] HybridClock initialized: now 1786173220319140 us; error 36 us; skew 500 ppm
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:40.321686 4374 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:13:40.322168 4374 webserver.cc:460] Webserver started at http://127.22.2.190:46001/ using document root <none> and password file <none>
I20260808 07:13:40.322417 4374 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:40.322468 4374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:40.401678 4374 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:40.402454 4393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.402627 4374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:40.402742 4374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "8b5fe885a6564a2b82e898a645939289"
format_stamp: "Formatted at 2026-08-08 07:13:38 on dist-test-slave-6k22"
server_key: "2cee7f631ba3a438418956f34fa7de8d"
server_key_iv: "5055287ecad6fa1e3faca43482427629"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:40.403111 4374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:40.417372 4374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:40.419445 4374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:40.419641 4374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:40.424043 4374 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:46817
I20260808 07:13:40.424104 4452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:46817 every 8 connection(s)
I20260808 07:13:40.424487 4374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:40.425733 4453 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:13:40.426405 4453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Bootstrap starting.
I20260808 07:13:40.431959 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 4374
I20260808 07:13:40.432264 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:51429/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:33849
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:46817
--webserver_port=41721
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:51429/kms with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:40.434742 4453 log.cc:824] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:40.436604 4453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:40.436928 4453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Bootstrap complete.
I20260808 07:13:40.439419 4453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } }
I20260808 07:13:40.439714 4453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5fe885a6564a2b82e898a645939289, State: Initialized, Role: FOLLOWER
I20260808 07:13:40.439924 4453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } }
I20260808 07:13:40.440011 4453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:40.440070 4453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:40.440126 4453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:13:40.441185 4453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } }
I20260808 07:13:40.441318 4453 leader_election.cc:302] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b5fe885a6564a2b82e898a645939289; no voters:
I20260808 07:13:40.441540 4453 leader_election.cc:288] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:13:40.441586 4459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:13:40.441707 4459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [term 2 LEADER]: Becoming Leader. State: Replica: 8b5fe885a6564a2b82e898a645939289, State: Running, Role: LEADER
I20260808 07:13:40.441848 4453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:40.441872 4459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } }
I20260808 07:13:40.442327 4461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b5fe885a6564a2b82e898a645939289. Latest consensus state: current_term: 2 leader_uuid: "8b5fe885a6564a2b82e898a645939289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } } }
I20260808 07:13:40.442430 4461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:40.442817 4460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8b5fe885a6564a2b82e898a645939289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5fe885a6564a2b82e898a645939289" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46817 } } }
I20260808 07:13:40.442895 4460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:40.443056 4466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:40.443590 4466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:40.443738 4466 catalog_manager.cc:1259] Loaded cluster ID: be41f7460d5c485a89cce4848e301edb
I20260808 07:13:40.443836 4466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:40.444021 4453 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:13:40.445103 4466 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:40.445284 4466 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Loaded TSK: 0
I20260808 07:13:40.445498 4466 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:13:40.544484 4456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:40.544765 4456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:40.544801 4456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:40.546574 4456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:40.546669 4456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:40.546730 4456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:13:40.548959 4456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:51429/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:33849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=41721
--tserver_master_addrs=127.22.2.190:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:40.549187 4456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:40.549487 4456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:40.556136 4496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.557044 4498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.557332 4495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:40.557638 4456 server_base.cc:1034] running on GCE node
I20260808 07:13:40.557932 4456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:40.558189 4456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:40.559367 4456 hybrid_clock.cc:648] HybridClock initialized: now 1786173220559358 us; error 72 us; skew 500 ppm
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:40.562434 4456 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:13:40.562893 4456 webserver.cc:460] Webserver started at http://127.22.2.129:41721/ using document root <none> and password file <none>
I20260808 07:13:40.563133 4456 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:40.563196 4456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:40.596040 4456 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260808 07:13:40.605772 4506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.606093 4456 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260808 07:13:40.606217 4456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "3ec85f1d70b64732830fbf530481810f"
format_stamp: "Formatted at 2026-08-08 07:13:39 on dist-test-slave-6k22"
server_key: "9350780e4366b28ddcc2a80e1f0960b9"
server_key_iv: "b7a4cc5889f23461818afa34fcf68d23"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:40.606778 4456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:40.626894 4456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:40.629285 4456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:40.629470 4456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:40.629756 4456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:40.630152 4456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:40.630209 4456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.630252 4456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:40.630280 4456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.637925 4456 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33849
I20260808 07:13:40.638021 4635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33849 every 8 connection(s)
I20260808 07:13:40.638497 4456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:40.641091 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 4456
I20260808 07:13:40.641436 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:51429/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:45127
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:46817
--webserver_port=34287
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:51429/kms with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:40.654356 4639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:40.639362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46817 (local address 127.22.2.129:33229)
0808 07:13:40.639579 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:40.639712 (+ 133us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:40.640322 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:40.640856 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:40.640862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:40.641158 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:40.641458 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:40.641466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.643425 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.643426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:40.644869 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:40.644872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.645737 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.645738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:40.645863 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:40.646425 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:40.646434 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:40.650223 (+ 3789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:40.652565 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:40.652580 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:40.652583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:40.652893 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:40.653181 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:40.653182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:40.653182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:40.653309 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:40.653716 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:40.653722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:40.653860 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 07:13:40.653963 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:13:40.654127 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260808 07:13:40.654342 4640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:40.639449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:33229 (local address 127.22.2.190:46817)
0808 07:13:40.639736 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:40.639744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:40.639811 (+ 67us) server_negotiation.cc:407] Connection header received
0808 07:13:40.640573 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:40.640579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:40.640628 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:40.640779 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:40.641618 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.643285 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.644975 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.645608 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.645750 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:40.650356 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:40.650361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:40.650361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:40.650375 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:40.652457 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:40.652991 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:40.652995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:40.652995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:40.653102 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:40.653389 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:40.653390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:40.653390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:40.653628 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:40.653723 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:40.654029 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 07:13:40.654142 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":110,"threads_started":1}
I20260808 07:13:40.655596 4636 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46817
I20260808 07:13:40.655692 4636 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:40.655884 4636 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
W20260808 07:13:40.656802 4636 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:46817 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:13:40.656883 4636 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:40.656944 4636 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:40.657399 4636 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:40.657473 4636 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
W20260808 07:13:40.657788 4636 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:13:40.769270 4641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:40.769559 4641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:40.769600 4641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:40.771268 4641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:40.771329 4641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:40.771373 4641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:13:40.773702 4641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:51429/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:45127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=34287
--tserver_master_addrs=127.22.2.190:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:40.774152 4641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:40.774466 4641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:40.784601 4648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.784852 4647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.784927 4650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:40.785081 4641 server_base.cc:1034] running on GCE node
I20260808 07:13:40.785336 4641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:40.785636 4641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:40.786844 4641 hybrid_clock.cc:648] HybridClock initialized: now 1786173220786826 us; error 39 us; skew 500 ppm
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:40.790186 4641 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:13:40.790740 4641 webserver.cc:460] Webserver started at http://127.22.2.130:34287/ using document root <none> and password file <none>
I20260808 07:13:40.791019 4641 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:40.791114 4641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:40.808305 4641 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:40.809420 4661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.809594 4641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:40.809684 4641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "6387d2795d6f4c54b1e96a5272e7a29d"
format_stamp: "Formatted at 2026-08-08 07:13:39 on dist-test-slave-6k22"
server_key: "e21fd2093aaebf646da17ffdd4504452"
server_key_iv: "7679c6ba8b45239830061b5afd1f4221"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:40.810155 4641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:40.837345 4641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:40.839821 4641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:40.839998 4641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:40.840247 4641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:40.840694 4641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:40.840749 4641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.840780 4641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:40.840819 4641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:40.848621 4641 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:45127
I20260808 07:13:40.848701 4792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:45127 every 8 connection(s)
I20260808 07:13:40.849179 4641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:13:40.849797 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 4641
I20260808 07:13:40.850106 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.2.148:51429/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:39473
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:46817
--webserver_port=38007
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:51429/kms with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:40.863951 4640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:40.852467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:47173 (local address 127.22.2.190:46817)
0808 07:13:40.852513 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:40.852515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:40.852528 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:13:40.853270 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:40.853272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:40.853290 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:40.853350 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:40.854251 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.855035 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.856877 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.857175 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.857262 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:40.860309 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:40.860310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:40.860310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:40.860319 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:40.862212 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:40.862712 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:40.862713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:40.862714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:40.862776 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:40.863072 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:40.863073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:40.863073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:40.863209 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:40.863261 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:40.863691 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 07:13:40.863799 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:13:40.864030 4797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:40.852152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46817 (local address 127.22.2.130:47173)
0808 07:13:40.852366 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:40.852419 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:40.853105 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:40.853555 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:40.853559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:40.853820 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:40.854111 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:40.854130 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.855135 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.855136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:40.856754 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:40.856758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:40.857325 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:40.857326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:40.857505 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:40.858212 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:40.858225 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:40.860193 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:40.862308 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:40.862314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:40.862316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:40.862614 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:40.862867 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:40.862867 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:40.862868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:40.862996 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:40.863315 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:40.863321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:40.863481 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:13:40.863635 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 07:13:40.863789 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260808 07:13:40.864900 4793 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46817
I20260808 07:13:40.865005 4793 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:40.865198 4793 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
W20260808 07:13:40.865936 4793 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:46817 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:13:40.866026 4793 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:40.866097 4793 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:40.866524 4793 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:40.866629 4793 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
W20260808 07:13:40.866953 4793 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:13:40.964327 4796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:40.964598 4796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:40.964639 4796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:40.965936 4796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:40.966010 4796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:40.966058 4796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:13:40.967629 4796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.2.148:51429/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:39473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=38007
--tserver_master_addrs=127.22.2.190:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:40.967941 4796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:40.968196 4796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:40.973224 4803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:40.973536 4804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:40.974824 4796 server_base.cc:1034] running on GCE node
W20260808 07:13:40.975687 4806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:40.975986 4796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:40.976280 4796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:40.992465 4796 hybrid_clock.cc:648] HybridClock initialized: now 1786173220992438 us; error 82 us; skew 500 ppm
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:40.995824 4796 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:13:40.996460 4796 webserver.cc:460] Webserver started at http://127.22.2.131:38007/ using document root <none> and password file <none>
I20260808 07:13:40.996716 4796 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:40.996786 4796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:13:40 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:41.011823 4796 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:13:41.012817 4816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:41.012987 4796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:41.013082 4796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "9b11d0150f424495b5dc2e0872821841"
format_stamp: "Formatted at 2026-08-08 07:13:40 on dist-test-slave-6k22"
server_key: "5c567925160ef2d75dc7b8e5cb9c1834"
server_key_iv: "1779cbe0e25eb2ae2dede496e850457a"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:41.013427 4796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:41.033058 4796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:41.035547 4796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:41.035785 4796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:41.036114 4796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:41.036507 4796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:41.036584 4796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:41.036631 4796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:41.036664 4796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:41.044027 4796 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:39473
I20260808 07:13:41.044656 4796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:13:41.045109 4947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:39473 every 8 connection(s)
I20260808 07:13:41.049885 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 4796
Aug 08 07:13:41 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173220, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:41.055367 4640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:41.045955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35525 (local address 127.22.2.190:46817)
0808 07:13:41.046036 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:41.046038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:41.046050 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:13:41.046527 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:41.046529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:41.046542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:41.046607 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:41.047247 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:41.047899 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:41.049073 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:41.049281 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:41.049378 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:41.052009 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:41.052011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:41.052012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:41.052020 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:41.053662 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:41.054196 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:41.054197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:41.054198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:41.054267 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:41.054569 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:41.054571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:41.054571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:41.054721 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:41.054790 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:41.055178 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 07:13:41.055272 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:13:41.055562 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:41.045593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:46817 (local address 127.22.2.131:35525)
0808 07:13:41.045823 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:41.045891 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:41.046404 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:41.046683 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:41.046686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:41.046916 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:41.047139 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:41.047144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:41.048010 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:41.048011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:41.048971 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:41.048974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:41.049414 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:41.049416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:41.049533 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:41.050140 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:41.050151 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:41.051919 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:41.053773 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:41.053780 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:41.053782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:41.054093 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:41.054348 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:41.054349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:41.054349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:41.054485 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:41.054819 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:41.054825 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:41.055007 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 07:13:41.055153 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 07:13:41.055305 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
I20260808 07:13:41.056160 4948 heartbeater.cc:347] Connected to a master server at 127.22.2.190:46817
I20260808 07:13:41.056264 4948 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:41.056470 4948 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
W20260808 07:13:41.057174 4948 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:46817 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:13:41.057250 4948 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:41.057317 4948 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:41.057756 4948 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:41.057839 4948 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
W20260808 07:13:41.058219 4948 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:13:41.065315 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:41.050499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.22.2.190:46817)
0808 07:13:41.050613 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:41.050615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:41.051550 (+ 935us) server_negotiation.cc:407] Connection header received
0808 07:13:41.051598 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:41.051599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:41.051613 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:41.051695 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:41.052455 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:41.053032 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:41.054560 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:41.054870 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:41.054941 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:41.059977 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:41.059978 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:41.059978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:41.059987 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:41.061768 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:41.064008 (+ 2240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:41.064010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:41.064010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:41.064078 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:41.064316 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:41.064317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:41.064317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:41.064450 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:41.064615 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:41.065088 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 07:13:41.065198 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
Aug 08 07:13:41 dist-test-slave-6k22 krb5kdc[3282](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173221, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:41.657974 4636 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:41.658144 4636 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:41.658586 4415 ts_manager.cc:194] Registered new tserver with Master: 3ec85f1d70b64732830fbf530481810f (127.22.2.129:33849)
I20260808 07:13:41.659521 4415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:33229
I20260808 07:13:41.867120 4793 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:41.867290 4793 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:41.867610 4415 ts_manager.cc:194] Registered new tserver with Master: 6387d2795d6f4c54b1e96a5272e7a29d (127.22.2.130:45127)
I20260808 07:13:41.868149 4415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:47173
Aug 08 07:13:41 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173221, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:42.058403 4948 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:42.058568 4948 heartbeater.cc:511] Master 127.22.2.190:46817 requested a full tablet report, sending...
I20260808 07:13:42.058872 4415 ts_manager.cc:194] Registered new tserver with Master: 9b11d0150f424495b5dc2e0872821841 (127.22.2.131:39473)
I20260808 07:13:42.059403 4415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:35525
I20260808 07:13:42.061827 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:42.071882 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.062856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.22.2.190:46817)
0808 07:13:42.063671 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.063673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.063682 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:42.063705 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.063706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.063718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.063796 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:42.064561 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.065336 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.066505 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.066824 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.066893 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.068552 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:42.068554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:42.068554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:42.068563 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:42.070493 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:42.070942 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:42.070943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:42.070943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:42.071012 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:42.071261 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:42.071262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:42.071262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:42.071402 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:42.071502 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.071682 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.071770 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":69,"threads_started":1}
I20260808 07:13:42.078171 4415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:13:42.080618 4415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:13:42.089802 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.083729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33849 (local address 127.0.0.1:40102)
0808 07:13:42.084330 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:42.084371 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:42.084436 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:42.084841 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:42.084845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:42.084848 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:42.084922 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.084925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.086350 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.086351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.087877 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.087881 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.089469 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.089470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.089560 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.089576 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:13:42.089654 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:13:42.089710 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":93,"threads_started":1}
I20260808 07:13:42.090286 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.084151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40102 (local address 127.22.2.129:33849)
0808 07:13:42.084663 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.084668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.084678 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:42.084711 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.084713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.084729 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.084806 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:42.085030 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.086163 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.087983 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.089268 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.089360 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.089428 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.090156 (+ 728us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.090202 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":75,"threads_started":1}
I20260808 07:13:42.091387 4566 tablet_service.cc:1467] Processing CreateTablet for tablet fd162470d45943e4aa1535b85ce516bc (DEFAULT_TABLE table=test-table [id=e8306718088e46008c51fcf57f783223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:42.091850 4566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd162470d45943e4aa1535b85ce516bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:42.095254 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.083743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:39473 (local address 127.0.0.1:39810)
0808 07:13:42.084533 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:42.084541 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:42.084573 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:42.088263 (+ 3690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:42.088263 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:42.088265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:42.088309 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.088310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.090777 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.090778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.091941 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.091943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.095024 (+ 3081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.095025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.095092 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.095097 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:13:42.095126 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 07:13:42.095165 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":103,"threads_started":1}
I20260808 07:13:42.095861 4979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.083966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:45127 (local address 127.0.0.1:42550)
0808 07:13:42.084908 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:42.084917 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:42.084953 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:42.087927 (+ 2974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:42.087928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:42.087930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:42.087974 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.087975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.092906 (+ 4931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.092907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.094016 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.094018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.095661 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.095662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.095725 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.095729 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:42.095761 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 07:13:42.095794 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":58,"threads_started":1}
I20260808 07:13:42.096182 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.084120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.22.2.130:45127)
0808 07:13:42.087687 (+ 3567us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.087692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.087704 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:13:42.087739 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.087741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.087757 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.087837 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:42.089599 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.090661 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.094122 (+ 3461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.094904 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.095964 (+ 1060us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.096034 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.096061 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.096105 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3527,"thread_start_us":59,"threads_started":1}
I20260808 07:13:42.096351 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.084083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.22.2.131:39473)
0808 07:13:42.088097 (+ 4014us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.088101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.088110 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:42.088141 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.088143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.088158 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.088373 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:42.088446 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.089499 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.092046 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.092797 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.095368 (+ 2571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.095442 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.095469 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.095513 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3978,"thread_start_us":94,"threads_started":1}
I20260808 07:13:42.096973 4712 tablet_service.cc:1467] Processing CreateTablet for tablet fd162470d45943e4aa1535b85ce516bc (DEFAULT_TABLE table=test-table [id=e8306718088e46008c51fcf57f783223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:42.097026 4875 tablet_service.cc:1467] Processing CreateTablet for tablet fd162470d45943e4aa1535b85ce516bc (DEFAULT_TABLE table=test-table [id=e8306718088e46008c51fcf57f783223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:42.097337 4712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd162470d45943e4aa1535b85ce516bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:42.097366 4875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd162470d45943e4aa1535b85ce516bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:42.101168 4988 tablet_bootstrap.cc:492] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f: Bootstrap starting.
I20260808 07:13:42.101845 4988 tablet_bootstrap.cc:654] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:42.102140 4988 log.cc:824] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:42.102476 4987 tablet_bootstrap.cc:492] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Bootstrap starting.
I20260808 07:13:42.103149 4987 tablet_bootstrap.cc:654] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:42.103436 4987 log.cc:824] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:42.104038 4988 tablet_bootstrap.cc:492] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f: No bootstrap required, opened a new log
I20260808 07:13:42.104156 4988 ts_tablet_manager.cc:1397] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:13:42.106863 4988 raft_consensus.cc:348] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.107038 4988 raft_consensus.cc:374] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:42.107070 4988 raft_consensus.cc:729] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec85f1d70b64732830fbf530481810f, State: Initialized, Role: FOLLOWER
I20260808 07:13:42.107290 4989 tablet_bootstrap.cc:492] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: Bootstrap starting.
I20260808 07:13:42.108031 4989 tablet_bootstrap.cc:654] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:42.108166 4987 tablet_bootstrap.cc:492] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: No bootstrap required, opened a new log
I20260808 07:13:42.108309 4987 ts_tablet_manager.cc:1397] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 07:13:42.108513 4989 log.cc:824] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:42.108959 4988 consensus_queue.cc:260] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.109338 4988 ts_tablet_manager.cc:1428] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260808 07:13:42.109496 4636 heartbeater.cc:503] Master 127.22.2.190:46817 was elected leader, sending a full tablet report...
I20260808 07:13:42.111465 4989 tablet_bootstrap.cc:492] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: No bootstrap required, opened a new log
I20260808 07:13:42.111605 4989 ts_tablet_manager.cc:1397] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 07:13:42.112795 4987 raft_consensus.cc:348] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.113034 4987 raft_consensus.cc:374] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:42.113152 4987 raft_consensus.cc:729] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b11d0150f424495b5dc2e0872821841, State: Initialized, Role: FOLLOWER
I20260808 07:13:42.113351 4987 consensus_queue.cc:260] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.113718 4987 ts_tablet_manager.cc:1428] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 07:13:42.113889 4948 heartbeater.cc:503] Master 127.22.2.190:46817 was elected leader, sending a full tablet report...
I20260808 07:13:42.114135 4989 raft_consensus.cc:348] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.114322 4989 raft_consensus.cc:374] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:42.114374 4989 raft_consensus.cc:729] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6387d2795d6f4c54b1e96a5272e7a29d, State: Initialized, Role: FOLLOWER
I20260808 07:13:42.114499 4989 consensus_queue.cc:260] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.114770 4989 ts_tablet_manager.cc:1428] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:13:42.114921 4793 heartbeater.cc:503] Master 127.22.2.190:46817 was elected leader, sending a full tablet report...
W20260808 07:13:42.139902 4638 tablet.cc:2369] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:42.263713 4995 raft_consensus.cc:482] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:42.263931 4995 raft_consensus.cc:504] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.264334 4995 leader_election.cc:288] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ec85f1d70b64732830fbf530481810f (127.22.2.129:33849), 6387d2795d6f4c54b1e96a5272e7a29d (127.22.2.130:45127)
I20260808 07:13:42.268251 4994 raft_consensus.cc:482] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:42.268384 4994 raft_consensus.cc:504] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.268805 4994 leader_election.cc:288] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b11d0150f424495b5dc2e0872821841 (127.22.2.131:39473), 6387d2795d6f4c54b1e96a5272e7a29d (127.22.2.130:45127)
Aug 08 07:13:42 dist-test-slave-6k22 krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173221, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for HTTP/127.22.2.148@KRBTEST.COM
I20260808 07:13:42.273155 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.265395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:41651 (local address 127.22.2.129:33849)
0808 07:13:42.265483 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.265485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.266859 (+ 1374us) server_negotiation.cc:407] Connection header received
0808 07:13:42.266973 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.266975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.266989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.267019 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:42.267298 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.268092 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.271249 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.272628 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.272681 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.272691 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.273040 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.273087 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:13:42.274019 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.264543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33849 (local address 127.22.2.131:41651)
0808 07:13:42.266773 (+ 2230us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:42.266785 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:42.266892 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:42.267088 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:42.267089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:42.267091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:42.267212 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.267214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.269790 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.269791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.271140 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.271143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.272864 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.272865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.272952 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.272956 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:42.273408 (+ 452us) client_negotiation.cc:240] Negotiation successful
0808 07:13:42.273456 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2200,"thread_start_us":154,"threads_started":1}
I20260808 07:13:42.274561 4587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd162470d45943e4aa1535b85ce516bc" candidate_uuid: "9b11d0150f424495b5dc2e0872821841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec85f1d70b64732830fbf530481810f" is_pre_election: true
I20260808 07:13:42.274791 4587 raft_consensus.cc:2454] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b11d0150f424495b5dc2e0872821841 in term 0.
I20260808 07:13:42.275312 4822 leader_election.cc:302] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec85f1d70b64732830fbf530481810f, 9b11d0150f424495b5dc2e0872821841; no voters:
I20260808 07:13:42.275466 4995 raft_consensus.cc:2781] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:42.275507 4995 raft_consensus.cc:482] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:42.275528 4995 raft_consensus.cc:3037] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:42.276752 4995 raft_consensus.cc:504] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.276867 4995 leader_election.cc:288] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [CANDIDATE]: Term 1 election: Requested vote from peers 3ec85f1d70b64732830fbf530481810f (127.22.2.129:33849), 6387d2795d6f4c54b1e96a5272e7a29d (127.22.2.130:45127)
I20260808 07:13:42.277169 4587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd162470d45943e4aa1535b85ce516bc" candidate_uuid: "9b11d0150f424495b5dc2e0872821841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec85f1d70b64732830fbf530481810f"
I20260808 07:13:42.277252 4587 raft_consensus.cc:3037] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:42.278697 4587 raft_consensus.cc:2454] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b11d0150f424495b5dc2e0872821841 in term 1.
I20260808 07:13:42.278930 4822 leader_election.cc:302] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec85f1d70b64732830fbf530481810f, 9b11d0150f424495b5dc2e0872821841; no voters:
I20260808 07:13:42.279850 4995 raft_consensus.cc:2781] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:42.280019 4995 raft_consensus.cc:686] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 1 LEADER]: Becoming Leader. State: Replica: 9b11d0150f424495b5dc2e0872821841, State: Running, Role: LEADER
I20260808 07:13:42.280122 4995 consensus_queue.cc:237] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } }
I20260808 07:13:42.283308 4415 catalog_manager.cc:5495] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b11d0150f424495b5dc2e0872821841 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "9b11d0150f424495b5dc2e0872821841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b11d0150f424495b5dc2e0872821841" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 } health_report { overall_health: UNKNOWN } } }
I20260808 07:13:42.284020 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.265055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:36405 (local address 127.22.2.130:45127)
0808 07:13:42.265218 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.265220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.274886 (+ 9666us) server_negotiation.cc:407] Connection header received
0808 07:13:42.274918 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.274919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.274933 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.274985 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:42.275234 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.275985 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.277252 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.277789 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.277968 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.277979 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.283883 (+ 5904us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.283950 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260808 07:13:42.285533 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.264815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:45127 (local address 127.22.2.131:36405)
0808 07:13:42.274390 (+ 9575us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:42.274401 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:42.274467 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:42.275124 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:42.275125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:42.275126 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:42.275151 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.275152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.276110 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.276111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.277153 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.277155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.278043 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.278044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.278109 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.278112 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:13:42.285414 (+ 7302us) client_negotiation.cc:240] Negotiation successful
0808 07:13:42.285466 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9563,"thread_start_us":124,"threads_started":1}
I20260808 07:13:42.286192 4739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd162470d45943e4aa1535b85ce516bc" candidate_uuid: "9b11d0150f424495b5dc2e0872821841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" is_pre_election: true
I20260808 07:13:42.286405 4739 raft_consensus.cc:2454] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b11d0150f424495b5dc2e0872821841 in term 0.
I20260808 07:13:42.286654 4738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd162470d45943e4aa1535b85ce516bc" candidate_uuid: "9b11d0150f424495b5dc2e0872821841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6387d2795d6f4c54b1e96a5272e7a29d"
I20260808 07:13:42.286775 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.269223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:60977 (local address 127.22.2.131:39473)
0808 07:13:42.269265 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.269267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.274672 (+ 5405us) server_negotiation.cc:407] Connection header received
0808 07:13:42.274790 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.274792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.274803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.274843 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:42.278243 (+ 3400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.279049 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.285688 (+ 6639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.286219 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.286271 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.286280 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.286655 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.286716 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:13:42.286761 4738 raft_consensus.cc:3037] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:42.287253 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.268994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:39473 (local address 127.22.2.129:60977)
0808 07:13:42.274595 (+ 5601us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:42.274607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:42.274698 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:42.274940 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:42.274941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:42.274943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:42.275057 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.275058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.279188 (+ 4130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.279189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.285322 (+ 6133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.285326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.286405 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.286407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.286498 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.286503 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:13:42.286966 (+ 463us) client_negotiation.cc:240] Negotiation successful
0808 07:13:42.287042 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5574,"thread_start_us":139,"threads_started":1}
I20260808 07:13:42.287674 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd162470d45943e4aa1535b85ce516bc" candidate_uuid: "3ec85f1d70b64732830fbf530481810f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b11d0150f424495b5dc2e0872821841" is_pre_election: true
I20260808 07:13:42.289497 4738 raft_consensus.cc:2454] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b11d0150f424495b5dc2e0872821841 in term 1.
I20260808 07:13:42.290907 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.269617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:50601 (local address 127.22.2.130:45127)
0808 07:13:42.284113 (+ 14496us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.284115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.287643 (+ 3528us) server_negotiation.cc:407] Connection header received
0808 07:13:42.287744 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.287745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.287754 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.287783 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:42.288140 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.288803 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.290058 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.290555 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.290602 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.290609 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.290813 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.290849 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14478,"thread_start_us":102,"threads_started":1}
I20260808 07:13:42.291253 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.269404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:45127 (local address 127.22.2.129:50601)
0808 07:13:42.287572 (+ 18168us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:42.287584 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:42.287673 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:42.287840 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:42.287841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:42.287842 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:42.287874 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.287875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.288901 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.288902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.289960 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:42.289963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.290665 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.290666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:42.290734 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.290738 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:42.291134 (+ 396us) client_negotiation.cc:240] Negotiation successful
0808 07:13:42.291186 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18139,"thread_start_us":94,"threads_started":1}
I20260808 07:13:42.291648 4738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd162470d45943e4aa1535b85ce516bc" candidate_uuid: "3ec85f1d70b64732830fbf530481810f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" is_pre_election: true
I20260808 07:13:42.291750 4738 raft_consensus.cc:2379] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ec85f1d70b64732830fbf530481810f in current term 1: Already voted for candidate 9b11d0150f424495b5dc2e0872821841 in this term.
I20260808 07:13:42.292044 4512 leader_election.cc:302] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ec85f1d70b64732830fbf530481810f; no voters: 6387d2795d6f4c54b1e96a5272e7a29d, 9b11d0150f424495b5dc2e0872821841
I20260808 07:13:42.292253 4994 raft_consensus.cc:2726] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 07:13:42.296351 4949 tablet.cc:2369] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:42.328920 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.325677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.22.2.131:39473)
0808 07:13:42.325749 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:42.325751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:42.325760 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:42.325804 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:42.325805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:42.325828 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:42.325892 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:42.326076 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.326798 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.327877 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:42.328154 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:42.328207 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:42.328394 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:42.328531 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:42.328601 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:42.328816 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 07:13:42.328840 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:13:42.331321 4738 raft_consensus.cc:1261] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 1 FOLLOWER]: Refusing update from remote peer 9b11d0150f424495b5dc2e0872821841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:42.331789 5006 consensus_queue.cc:1035] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6387d2795d6f4c54b1e96a5272e7a29d" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:42.334586 4587 raft_consensus.cc:1261] T fd162470d45943e4aa1535b85ce516bc P 3ec85f1d70b64732830fbf530481810f [term 1 FOLLOWER]: Refusing update from remote peer 9b11d0150f424495b5dc2e0872821841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:42.334959 5006 consensus_queue.cc:1035] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec85f1d70b64732830fbf530481810f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:42.340711 5010 mvcc.cc:205] Tried to move back new op lower bound from 7316165518665875456 to 7316165518459953152. Current Snapshot: MvccSnapshot[applied={T|T < 7316165518665875456}]
I20260808 07:13:42.341053 5013 mvcc.cc:205] Tried to move back new op lower bound from 7316165518665875456 to 7316165518459953152. Current Snapshot: MvccSnapshot[applied={T|T < 7316165518665875456}]
I20260808 07:13:42.344749 5011 mvcc.cc:205] Tried to move back new op lower bound from 7316165518665875456 to 7316165518459953152. Current Snapshot: MvccSnapshot[applied={T|T < 7316165518665875456}]
I20260808 07:13:42.346472 4415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:42.346575 4415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:42.347460 4415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b5fe885a6564a2b82e898a645939289: Sending DeleteTablet for 3 replicas of tablet fd162470d45943e4aa1535b85ce516bc
I20260808 07:13:42.347983 4712 tablet_service.cc:1514] Processing DeleteTablet for tablet fd162470d45943e4aa1535b85ce516bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:42 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:42550
I20260808 07:13:42.347996 4566 tablet_service.cc:1514] Processing DeleteTablet for tablet fd162470d45943e4aa1535b85ce516bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:42 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40102
I20260808 07:13:42.348735 4875 tablet_service.cc:1514] Processing DeleteTablet for tablet fd162470d45943e4aa1535b85ce516bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:42 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:39810
I20260808 07:13:42.349428 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 4456
I20260808 07:13:42.358925 5021 tablet_replica.cc:330] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: stopping tablet replica
I20260808 07:13:42.359155 5021 raft_consensus.cc:2229] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:13:42.359417 5021 raft_consensus.cc:2258] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:42.360008 5021 ts_tablet_manager.cc:1905] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:42.362286 5021 ts_tablet_manager.cc:1918] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:42.362434 5021 log.cc:1196] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/fd162470d45943e4aa1535b85ce516bc
I20260808 07:13:42.362814 5021 ts_tablet_manager.cc:1939] T fd162470d45943e4aa1535b85ce516bc P 9b11d0150f424495b5dc2e0872821841: Deleting consensus metadata
I20260808 07:13:42.363381 4397 catalog_manager.cc:4841] TS 9b11d0150f424495b5dc2e0872821841 (127.22.2.131:39473): tablet fd162470d45943e4aa1535b85ce516bc (table test-table [id=e8306718088e46008c51fcf57f783223]) successfully deleted
I20260808 07:13:42.363730 5022 tablet_replica.cc:330] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: stopping tablet replica
I20260808 07:13:42.363867 5022 raft_consensus.cc:2229] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 07:13:42.363984 4400 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:33849 (error 108)
I20260808 07:13:42.364025 5022 raft_consensus.cc:2258] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:42.364535 5022 ts_tablet_manager.cc:1905] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:42.366676 5022 ts_tablet_manager.cc:1918] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:42.366763 5022 log.cc:1196] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/fd162470d45943e4aa1535b85ce516bc
I20260808 07:13:42.367067 5022 ts_tablet_manager.cc:1939] T fd162470d45943e4aa1535b85ce516bc P 6387d2795d6f4c54b1e96a5272e7a29d: Deleting consensus metadata
I20260808 07:13:42.367658 4400 catalog_manager.cc:4841] TS 6387d2795d6f4c54b1e96a5272e7a29d (127.22.2.130:45127): tablet fd162470d45943e4aa1535b85ce516bc (table test-table [id=e8306718088e46008c51fcf57f783223]) successfully deleted
I20260808 07:13:42.385756 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:42.385595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33849 (local address 127.0.0.1:40110)
0808 07:13:42.385657 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:42.385701 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:13:42.385812 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 4641
W20260808 07:13:42.387295 4400 catalog_manager.cc:4584] TS 3ec85f1d70b64732830fbf530481810f (127.22.2.129:33849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd162470d45943e4aa1535b85ce516bc: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33849: connect: Connection refused (error 111)
I20260808 07:13:42.397534 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 4796
I20260808 07:13:42.418495 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 4374
I20260808 07:13:42.425791 22538 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:13:42.769955 22538 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:13:42.770078 22538 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:13:43.114632 22538 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:13:43.114 UTC [3317] LOG: received smart shutdown request
2026-08-08 07:13:43.119 UTC [3317] LOG: background worker "logical replication launcher" (PID 3328) exited with exit code 1
2026-08-08 07:13:43.119 UTC [3321] LOG: shutting down
2026-08-08 07:13:43.308 UTC [3317] LOG: database system is shut down
2026-08-08T07:13:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49508 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:13:43 dist-test-slave-6k22 krb5kdc[5033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:13:43 dist-test-slave-6k22 krb5kdc[5033](info): set up 2 sockets
Aug 08 07:13:43 dist-test-slave-6k22 krb5kdc[5033](info): commencing operation
krb5kdc: starting...
W20260808 07:13:44.526587 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:13:44 dist-test-slave-6k22 krb5kdc[5033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:13:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:13:44.655086 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40623
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40623,127.22.2.189:44335,127.22.2.188:35077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:44.738233 5048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:44.738502 5048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:44.738539 5048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:44.739730 5048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:44.739801 5048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:44.739826 5048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:44.739856 5048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:44.739888 5048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:44.741394 5048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40623,127.22.2.189:44335,127.22.2.188:35077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:44.741675 5048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:44.741906 5048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:44.746981 5055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:44.746968 5059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:44.746968 5057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:44.747201 5048 server_base.cc:1034] running on GCE node
I20260808 07:13:44.747516 5048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:44.747836 5048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:44.749002 5048 hybrid_clock.cc:648] HybridClock initialized: now 1786173224748983 us; error 40 us; skew 500 ppm
Aug 08 07:13:44 dist-test-slave-6k22 krb5kdc[5033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:44.751478 5048 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:13:44.751972 5048 webserver.cc:460] Webserver started at http://127.22.2.190:42535/ using document root <none> and password file <none>
I20260808 07:13:44.752226 5048 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:44.752293 5048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:44.752449 5048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:44.753512 5048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "be451b335b634c8db60f99754f557315"
format_stamp: "Formatted at 2026-08-08 07:13:44 on dist-test-slave-6k22"
server_key: "4d20513c5c4eccdb07cfa3c219cae29c"
server_key_iv: "56950c5a2f7c19fda491d0b80eed5d64"
server_key_version: "encryptionkey@0"
I20260808 07:13:44.753892 5048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "be451b335b634c8db60f99754f557315"
format_stamp: "Formatted at 2026-08-08 07:13:44 on dist-test-slave-6k22"
server_key: "4d20513c5c4eccdb07cfa3c219cae29c"
server_key_iv: "56950c5a2f7c19fda491d0b80eed5d64"
server_key_version: "encryptionkey@0"
I20260808 07:13:44.755223 5048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:13:44.756088 5066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:44.756276 5048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:44.756395 5048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "be451b335b634c8db60f99754f557315"
format_stamp: "Formatted at 2026-08-08 07:13:44 on dist-test-slave-6k22"
server_key: "4d20513c5c4eccdb07cfa3c219cae29c"
server_key_iv: "56950c5a2f7c19fda491d0b80eed5d64"
server_key_version: "encryptionkey@0"
I20260808 07:13:44.756469 5048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:44.781864 5048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:44.784101 5048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:44.784271 5048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:44.788839 5048 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:40623
I20260808 07:13:44.788878 5128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:40623 every 8 connection(s)
I20260808 07:13:44.789254 5048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:44.790164 5129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:44.790680 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 5048
I20260808 07:13:44.790843 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:13:44.791059 22538 external_mini_cluster.cc:1428] Setting key 670a7b167664e6f12de589e833e0c8b6
I20260808 07:13:44.792701 5129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.2.189@KRBTEST.COM; defaulting to no policy
Aug 08 07:13:44 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.189@KRBTEST.COM" created.
I20260808 07:13:44.804008 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.793613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40623 (local address 127.0.0.1:60542)
0808 07:13:44.793795 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.793838 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:44.794310 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:44.794661 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:44.794665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:44.795062 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:44.795314 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:44.795321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.796513 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.796514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:44.797717 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:44.797720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.798262 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.798263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:44.798364 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:44.798938 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:44.798949 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:44.800547 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:44.802393 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:44.802399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:44.802401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:44.802586 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:44.802928 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:44.802929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:44.802929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:44.803044 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:44.803499 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:44.803505 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:44.803605 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 07:13:44.803697 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:13:44.803840 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260808 07:13:44.804133 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.793664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.22.2.190:40623)
0808 07:13:44.794447 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:44.794450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:44.794461 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:13:44.794519 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:44.794520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:44.794535 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:44.794596 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:44.795471 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.796408 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.797827 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.798155 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.798249 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:44.800675 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:44.800683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:44.800683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:44.800689 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:44.802316 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:44.802658 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:44.802661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:44.802661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:44.802849 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:44.803117 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:44.803117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:44.803118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:44.803420 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:44.803510 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:44.803782 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 07:13:44.803935 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":763,"thread_start_us":111,"threads_started":1}
I20260808 07:13:44.805493 5129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:44.805850 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.805742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53412)
0808 07:13:44.805790 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.805833 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:13:44.806032 5070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
I20260808 07:13:44.806427 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.806362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53416)
0808 07:13:44.806387 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.806411 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 07:13:44.806545 5129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:44335: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
I20260808 07:13:44.818962 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:44335
--webserver_interface=127.22.2.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40623,127.22.2.189:44335,127.22.2.188:35077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:44.855785 5129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } attempt: 1
I20260808 07:13:44.856333 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.856164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53432)
0808 07:13:44.856244 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.856284 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 07:13:44.856699 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.856600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53440)
0808 07:13:44.856648 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.856675 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:13:44.856784 5129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:44335: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
W20260808 07:13:44.902522 5136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:44.902736 5136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:44.902760 5136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:44.903939 5136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:44.903983 5136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:44.903997 5136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:44.904011 5136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:44.904022 5136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:44.905512 5136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40623,127.22.2.189:44335,127.22.2.188:35077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:44335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.2.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:44.905769 5136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:44.906023 5136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:44.910909 5143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:44.910916 5146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:44.910948 5144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:44.911147 5136 server_base.cc:1034] running on GCE node
I20260808 07:13:44.911473 5136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:44.911904 5136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:44.913121 5136 hybrid_clock.cc:648] HybridClock initialized: now 1786173224913095 us; error 39 us; skew 500 ppm
Aug 08 07:13:44 dist-test-slave-6k22 krb5kdc[5033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:44.915761 5136 init.cc:377] Logged in from keytab as kudu/127.22.2.189@KRBTEST.COM (short username kudu)
I20260808 07:13:44.916148 5136 webserver.cc:460] Webserver started at http://127.22.2.189:41105/ using document root <none> and password file <none>
I20260808 07:13:44.916334 5136 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:44.916374 5136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:44.916463 5136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:44.917516 5136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/data/instance:
uuid: "67f1477558ab4184be2dd31ae3c950dd"
format_stamp: "Formatted at 2026-08-08 07:13:44 on dist-test-slave-6k22"
server_key: "3ec3c3907cee29313f78203d646a79fc"
server_key_iv: "a22c84be620aa990e8519f570c5cea45"
server_key_version: "encryptionkey@0"
I20260808 07:13:44.917825 5136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/wal/instance:
uuid: "67f1477558ab4184be2dd31ae3c950dd"
format_stamp: "Formatted at 2026-08-08 07:13:44 on dist-test-slave-6k22"
server_key: "3ec3c3907cee29313f78203d646a79fc"
server_key_iv: "a22c84be620aa990e8519f570c5cea45"
server_key_version: "encryptionkey@0"
I20260808 07:13:44.919088 5136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:13:44.919838 5157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:44.920017 5136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:44.920104 5136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/wal
uuid: "67f1477558ab4184be2dd31ae3c950dd"
format_stamp: "Formatted at 2026-08-08 07:13:44 on dist-test-slave-6k22"
server_key: "3ec3c3907cee29313f78203d646a79fc"
server_key_iv: "a22c84be620aa990e8519f570c5cea45"
server_key_version: "encryptionkey@0"
I20260808 07:13:44.920158 5136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:44.924922 5129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } attempt: 2
I20260808 07:13:44.925339 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.925228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53448)
0808 07:13:44.925273 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.925311 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 07:13:44.925758 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.925658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53460)
0808 07:13:44.925711 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.925742 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:13:44.925860 5129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:44335: Network error: Client connection negotiation failed: client connection to 127.22.2.189:44335: connect: Connection refused (error 111)
I20260808 07:13:44.939957 5136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:44.942224 5136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:44.942345 5136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:44.946528 5136 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:44335
I20260808 07:13:44.946579 5214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:44335 every 8 connection(s)
I20260808 07:13:44.946959 5136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
I20260808 07:13:44.948004 5215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:44.949563 5215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:44.954654 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 5136
I20260808 07:13:44.954778 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-1/wal/instance
I20260808 07:13:44.954960 22538 external_mini_cluster.cc:1428] Setting key 14e9e9ba56c4031b15520a174e4053d6
Aug 08 07:13:44 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:44.958882 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.950268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.22.2.190:40623)
0808 07:13:44.950326 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:44.950328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:44.950537 (+ 209us) server_negotiation.cc:407] Connection header received
0808 07:13:44.950964 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:44.950965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:44.950976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:44.951034 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:44.951752 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.952231 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.953362 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.953557 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.953609 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:44.955776 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:44.955778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:44.955778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:44.955787 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:44.957287 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:44.957805 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:44.957806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:44.957806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:44.957856 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:44.958138 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:44.958139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:44.958140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:44.958284 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:44.958385 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:44.958652 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:13:44.958745 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:13:44.959038 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.950225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40623 (local address 127.0.0.1:60554)
0808 07:13:44.950391 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.950433 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:44.950885 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:44.951143 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:44.951146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:44.951396 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:44.951641 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:44.951646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.952378 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.952380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:44.953285 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:44.953287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.953634 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.953635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:44.953771 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:44.954206 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:44.954213 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:44.955686 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:44.957379 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:44.957387 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:44.957389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:44.957725 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:44.957921 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:44.957922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:44.957923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:44.958040 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:44.958390 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:44.958398 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:44.958518 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 07:13:44.958603 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:13:44.958832 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260808 07:13:44.959729 5215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.2.188@KRBTEST.COM; defaulting to no policy
Aug 08 07:13:44 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.188@KRBTEST.COM" created.
I20260808 07:13:44.968005 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.960093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.22.2.189:44335)
0808 07:13:44.960284 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:44.960289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:44.960306 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:13:44.960325 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:44.960329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:44.960345 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:44.960428 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:44.960688 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.961395 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.962234 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.962738 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.962826 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:44.964871 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:44.964879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:44.964879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:44.964886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:44.966512 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:44.967038 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:44.967041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:44.967041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:44.967096 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:44.967293 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:44.967293 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:44.967294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:44.967482 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:44.967547 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:44.967798 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 07:13:44.967914 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":163,"thread_start_us":136,"threads_started":1}
I20260808 07:13:44.968019 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.960018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53470)
0808 07:13:44.960061 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.960070 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:44.960107 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:44.960429 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:44.960430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:44.960563 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:44.960609 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:44.960610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.961503 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.961504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:44.962150 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:44.962152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:44.962826 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:44.962827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:44.962909 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:44.963319 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:44.963323 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:44.964781 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:44.966686 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:44.966687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:44.966688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:44.966941 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:44.967167 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:44.967168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:44.967168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:44.967235 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:44.967597 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:44.967599 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:44.967648 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:13:44.967727 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:13:44.967850 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 07:13:44.968777 5215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:44.969122 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.969018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51006)
0808 07:13:44.969057 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.969102 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 07:13:44.969295 5158 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
I20260808 07:13:44.969816 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:44.969675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51020)
0808 07:13:44.969741 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:44.969782 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 07:13:44.969946 5215 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:35077: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
I20260808 07:13:44.987844 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:35077
--webserver_interface=127.22.2.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40623,127.22.2.189:44335,127.22.2.188:35077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:45.017025 5129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } attempt: 3
I20260808 07:13:45.019192 5215 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } attempt: 1
I20260808 07:13:45.019652 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.019489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51028)
0808 07:13:45.019547 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.019612 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:13:45.020161 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.020004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51030)
0808 07:13:45.020078 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.020119 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 07:13:45.020382 5215 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:35077: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 07:13:45.025420 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.017598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.22.2.189:44335)
0808 07:13:45.017716 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.017718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.017728 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:45.017754 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.017755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.017767 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.017844 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.018272 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.018834 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.019961 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.020273 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.020349 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.022566 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.022568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.022568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.022576 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.024241 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.024587 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.024588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.024589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.024649 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.024881 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.024882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.024882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.024999 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.025071 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.025252 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.025328 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:13:45.025427 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.017517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53472)
0808 07:13:45.017598 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.017612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:45.017664 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:45.017877 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:45.017879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:45.018092 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:45.018186 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.018189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.018923 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.018924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.019863 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.019867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.020362 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.020364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.020454 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.020951 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:45.020957 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:45.022473 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:45.024326 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.024328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.024328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.024510 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.024745 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.024746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.024747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.024815 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.025076 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:45.025078 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:45.025124 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:13:45.025223 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 07:13:45.025328 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 07:13:45.025840 5129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:45.026189 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.026090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51044)
0808 07:13:45.026141 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.026169 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:13:45.026531 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.026422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51048)
0808 07:13:45.026483 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.026507 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 07:13:45.026625 5129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:35077: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
I20260808 07:13:45.057783 5129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } attempt: 1
I20260808 07:13:45.058359 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.058178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51060)
0808 07:13:45.058266 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.058301 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 07:13:45.058795 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.058686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51064)
0808 07:13:45.058739 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.058774 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:13:45.058894 5129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:35077: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
W20260808 07:13:45.081532 5222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:45.081759 5222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:45.081784 5222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:45.083060 5222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:45.083143 5222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:45.083173 5222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:45.083197 5222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:45.083235 5222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:45.084887 5222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40623,127.22.2.189:44335,127.22.2.188:35077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:35077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.2.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:45.085094 5222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:45.085281 5222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:13:45.088553 5215 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } attempt: 2
I20260808 07:13:45.089119 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.088987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51078)
0808 07:13:45.089039 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.089080 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 07:13:45.089619 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.089492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51088)
0808 07:13:45.089546 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.089574 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:13:45.089807 5215 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:35077: Network error: Client connection negotiation failed: client connection to 127.22.2.188:35077: connect: Connection refused (error 111)
W20260808 07:13:45.090279 5231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:45.090345 5228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:45.090282 5229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:45.090495 5222 server_base.cc:1034] running on GCE node
I20260808 07:13:45.090758 5222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:45.091046 5222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:45.092221 5222 hybrid_clock.cc:648] HybridClock initialized: now 1786173225092213 us; error 35 us; skew 500 ppm
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173225, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:45.094739 5222 init.cc:377] Logged in from keytab as kudu/127.22.2.188@KRBTEST.COM (short username kudu)
I20260808 07:13:45.095118 5222 webserver.cc:460] Webserver started at http://127.22.2.188:42301/ using document root <none> and password file <none>
I20260808 07:13:45.095320 5222 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:45.095362 5222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:45.095451 5222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:45.096551 5222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/data/instance:
uuid: "7fa7f0210c134378b07590568096bc75"
format_stamp: "Formatted at 2026-08-08 07:13:45 on dist-test-slave-6k22"
server_key: "c63b00bc0b348c06f17f8d9825030bdb"
server_key_iv: "b0fa721d4303dd2b134146f21abb6ada"
server_key_version: "encryptionkey@0"
I20260808 07:13:45.096865 5222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/wal/instance:
uuid: "7fa7f0210c134378b07590568096bc75"
format_stamp: "Formatted at 2026-08-08 07:13:45 on dist-test-slave-6k22"
server_key: "c63b00bc0b348c06f17f8d9825030bdb"
server_key_iv: "b0fa721d4303dd2b134146f21abb6ada"
server_key_version: "encryptionkey@0"
I20260808 07:13:45.098101 5222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:45.098755 5238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:45.098973 5222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:45.099079 5222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/wal
uuid: "7fa7f0210c134378b07590568096bc75"
format_stamp: "Formatted at 2026-08-08 07:13:45 on dist-test-slave-6k22"
server_key: "c63b00bc0b348c06f17f8d9825030bdb"
server_key_iv: "b0fa721d4303dd2b134146f21abb6ada"
server_key_version: "encryptionkey@0"
I20260808 07:13:45.099171 5222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:45.111665 5222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:45.113624 5222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:45.113801 5222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:45.117928 5222 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:35077
I20260808 07:13:45.117983 5303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:35077 every 8 connection(s)
I20260808 07:13:45.118389 5222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
I20260808 07:13:45.119323 5304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:45.120947 5304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:45.123798 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 5222
I20260808 07:13:45.123919 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173068742558-22538-0/minicluster-data/master-2/wal/instance
I20260808 07:13:45.124126 22538 external_mini_cluster.cc:1428] Setting key ec112a96211ea62cdb55a7b20f2921f1
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173225, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 07:13:45.132143 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.121755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.22.2.190:40623)
0808 07:13:45.121835 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.121837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.121955 (+ 118us) server_negotiation.cc:407] Connection header received
0808 07:13:45.122402 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.122403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.122413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.122479 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.123121 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.123644 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.125131 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.125331 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.125388 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.128155 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.128157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.128157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.128174 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.130542 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.131002 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.131004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.131004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.131069 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.131377 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.131378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.131379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.131509 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.131616 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.131937 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.132035 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:13:45.132403 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.121659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:40623 (local address 127.0.0.1:60562)
0808 07:13:45.121823 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.121866 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:45.122315 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:45.122530 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:45.122533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:45.122785 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:45.123043 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.123047 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.123753 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.123754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.125041 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.125045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.125433 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.125434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.125601 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.126119 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:45.126128 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:45.128058 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:45.130648 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.130656 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.130658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.130914 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.131150 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.131151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.131151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.131298 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.131614 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:45.131623 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:45.131778 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 07:13:45.131897 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:13:45.132099 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260808 07:13:45.133028 5304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:45.133936 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.124511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.22.2.188:35077)
0808 07:13:45.124709 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.124712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.124727 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:13:45.124750 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.124752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.124768 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.124827 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.125579 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.126623 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.127892 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.128169 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.128254 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.130593 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.130601 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.130601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.130608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.132184 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.132640 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.132643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.132643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.132717 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.133001 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.133002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.133002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.133187 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.133339 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.133462 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.133814 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260808 07:13:45.134688 5129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } attempt: 2
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173225, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:45.141512 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.133455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.22.2.189:44335)
0808 07:13:45.133511 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.133513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.133522 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:45.133548 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.133549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.133559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.133648 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.133900 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.134568 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.135625 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.135958 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.136037 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.138551 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.138553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.138553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.138560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.140255 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.140664 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.140666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.140666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.140725 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.140959 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.140961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.140961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.141092 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.141137 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.141337 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.141416 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:13:45.141512 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.133342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:44335 (local address 127.0.0.1:53486)
0808 07:13:45.133406 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.133418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:45.133459 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:45.133651 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:45.133652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:45.133786 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:45.133816 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.133818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.134650 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.134651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.135447 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.135449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.136038 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.136038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.136082 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.137120 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:45.137125 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:45.138437 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:45.140346 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.140347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.140347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.140574 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.140802 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.140803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.140803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.140880 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.141192 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:45.141193 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:45.141236 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:13:45.141304 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:13:45.141423 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:13:45.141913 5304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:45.143718 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.134938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51110)
0808 07:13:45.135020 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.135033 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:45.135074 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:45.135215 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:45.135216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:45.135366 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:45.135399 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.135400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.136366 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.136367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.136993 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.136996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.137379 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.137380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.137443 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.137897 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:45.137902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:45.139549 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:45.142453 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.142455 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.142455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.142725 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.142950 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.142951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.142952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.143016 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.143381 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:45.143382 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:45.143428 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:13:45.143478 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:13:45.143595 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:13:45.144088 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.135071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.22.2.188:35077)
0808 07:13:45.135112 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.135114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.135122 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:13:45.135141 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.135142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.135150 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.135198 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.135493 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.136278 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.137077 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.137293 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.137381 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.139682 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.139683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.139683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.139691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.142344 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.142799 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.142800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.142800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.142861 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.143097 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.143097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.143098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.143242 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.143855 (+ 613us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.143907 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.143989 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:13:45.144796 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.134748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60566 (local address 127.22.2.190:40623)
0808 07:13:45.134789 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.134791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.134799 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:13:45.134821 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.134821 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.134859 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.134903 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.136095 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.136677 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.137897 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.138207 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.138284 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.140944 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.140945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.140946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.140952 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.143121 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.143504 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.143505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.143505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.143592 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.143918 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.143919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.143920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.144057 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.144371 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.144611 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.144698 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:13:45.145748 5129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315: Bootstrap starting.
I20260808 07:13:45.146443 5129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:45.146768 5129 log.cc:824] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315: Log is configured to *not* fsync() on all Append() calls
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173225, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 07:13:45.147739 5129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315: No bootstrap required, opened a new log
I20260808 07:13:45.149139 5129 raft_consensus.cc:348] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.149273 5129 raft_consensus.cc:374] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:45.149331 5129 raft_consensus.cc:729] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be451b335b634c8db60f99754f557315, State: Initialized, Role: FOLLOWER
I20260808 07:13:45.149451 5129 consensus_queue.cc:260] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.149722 5312 sys_catalog.cc:455] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.149843 5312 sys_catalog.cc:458] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:45.149787 5129 sys_catalog.cc:564] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:45.150401 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.142204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51120)
0808 07:13:45.142232 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.142241 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:45.142279 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:45.142493 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:45.142494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:45.142624 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:45.142647 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.142648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.144114 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.144115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.144800 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.144803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.145401 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.145403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.145448 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.145937 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:45.145942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:45.147271 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:45.149214 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.149215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.149215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.149458 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.149668 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.149668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.149669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.149728 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.150077 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:45.150078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:45.150119 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 07:13:45.150172 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:13:45.150264 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260808 07:13:45.150774 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.142234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.22.2.188:35077)
0808 07:13:45.142382 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.142383 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.142389 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:13:45.142404 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.142404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.142412 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.142490 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.142729 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.144028 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.145025 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.145318 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.145387 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.147434 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.147435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.147436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.147442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.149106 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.149521 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.149522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.149522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.149583 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.149810 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.149811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.149811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.149991 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.150517 (+ 526us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.150562 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.150646 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":109,"threads_started":1}
W20260808 07:13:45.151751 5324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:13:45.151868 5324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:13:45.152767 5304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75: Bootstrap starting.
I20260808 07:13:45.153479 5304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:45.153846 5304 log.cc:824] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:45.154632 5304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75: No bootstrap required, opened a new log
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 07:13:45.152899 5048 master_runner.cc:186] Error getting master registration for 127.22.2.189:44335: OK, instance_id { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" instance_seqno: 1786173224945333 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:13:45.155869 5048 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:13:45.156721 5304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.156875 5304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:45.156914 5304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa7f0210c134378b07590568096bc75, State: Initialized, Role: FOLLOWER
I20260808 07:13:45.157030 5304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.157364 5304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:45.157327 5327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.157527 5327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:45.159238 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.147744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.22.2.189:44335)
0808 07:13:45.149847 (+ 2103us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.149849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.149859 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:45.149881 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.149882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.149893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.149951 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.150637 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.151267 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.152384 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.152897 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.152974 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.155943 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.155944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.155945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.155952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.157810 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.158264 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.158266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.158266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.158326 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.158578 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.158579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.158579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.158771 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.158981 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.159024 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.159130 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2080}
W20260808 07:13:45.160032 5340 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:13:45.160094 5340 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:13:45.160924 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:13:45.162539 5222 master_runner.cc:186] Error getting master registration for 127.22.2.189:44335: OK, instance_id { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" instance_seqno: 1786173224945333 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:13:45.164891 5222 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:13:45.176585 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.162455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.22.2.190:40623)
0808 07:13:45.162500 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.162502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.167845 (+ 5343us) server_negotiation.cc:407] Connection header received
0808 07:13:45.167918 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.167919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.167933 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.167994 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.168894 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.169774 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.170913 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.171239 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.171285 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.173115 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.173116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.173117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.173124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.175022 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.175469 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.175471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.175471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.175537 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.175837 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.175838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.175839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.176007 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.176096 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.176355 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.176442 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:13:45.181017 5215 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } attempt: 3
I20260808 07:13:45.181985 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.163976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51124 (local address 127.22.2.188:35077)
0808 07:13:45.164015 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.164017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.168125 (+ 4108us) server_negotiation.cc:407] Connection header received
0808 07:13:45.168683 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.168684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.168698 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.168732 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.169885 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.170742 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.174756 (+ 4014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.175196 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.175241 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.178044 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.178045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.178046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.178054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.180430 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.180909 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.180910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.180911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.180970 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.181201 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.181202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.181203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.181373 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.181481 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.181778 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.181869 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:13:45.184195 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.162800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.22.2.189:44335)
0808 07:13:45.162838 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.162840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.168019 (+ 5179us) server_negotiation.cc:407] Connection header received
0808 07:13:45.168324 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.168325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.168337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.168372 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.170853 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.171627 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.173801 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.174156 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.174217 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.178255 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.178257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.178257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.178266 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.180324 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.180873 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.180874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.180875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.180957 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.183515 (+ 2558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.183516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.183517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.183675 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.183763 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.183976 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.184056 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 07:13:45 dist-test-slave-6k22 krb5kdc[5033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173224, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 07:13:45.189690 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.181594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51140 (local address 127.22.2.188:35077)
0808 07:13:45.181638 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:45.181640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:45.181648 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:13:45.181670 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:45.181670 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:45.181681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:45.181731 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:45.182556 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.183093 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.184184 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.184505 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.184573 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.186508 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:45.186510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:45.186510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:45.186518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:45.188486 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.188853 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.188855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.188855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.188921 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:45.189140 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:45.189141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:45.189141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:45.189266 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:45.189343 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:45.189516 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 07:13:45.189595 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:13:45.189709 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:45.181324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:35077 (local address 127.0.0.1:51140)
0808 07:13:45.181453 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:45.181465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:45.181506 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:45.181805 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:45.181806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:45.181973 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:45.182019 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.182022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.183208 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.183210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.184092 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:45.184094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:45.184629 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:45.184630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:45.184705 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:45.185164 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:45.185170 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:45.186412 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:45.188580 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.188581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.188581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.188772 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.189002 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:45.189003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:45.189003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:45.189070 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:45.189349 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:45.189351 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:45.189396 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:13:45.189489 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 07:13:45.189594 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 07:13:45.191479 5215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd: Bootstrap starting.
I20260808 07:13:45.192299 5215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:45.192626 5215 log.cc:824] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:45.193575 5215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd: No bootstrap required, opened a new log
I20260808 07:13:45.195322 5215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.195463 5215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:45.195513 5215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67f1477558ab4184be2dd31ae3c950dd, State: Initialized, Role: FOLLOWER
I20260808 07:13:45.195761 5215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.196065 5351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.196131 5215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:45.196238 5351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:45.200701 5136 master_runner.cc:263] No leader master found from master 67f1477558ab4184be2dd31ae3c950dd
I20260808 07:13:45.200750 5136 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 07:13:45.200899 5363 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:13:45.200994 5363 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:13:45.245045 5327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:45.245188 5327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.245406 5327 leader_election.cc:288] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be451b335b634c8db60f99754f557315 (127.22.2.190:40623), 67f1477558ab4184be2dd31ae3c950dd (127.22.2.189:44335)
I20260808 07:13:45.245850 5100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7fa7f0210c134378b07590568096bc75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be451b335b634c8db60f99754f557315" is_pre_election: true
I20260808 07:13:45.246001 5100 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fa7f0210c134378b07590568096bc75 in term 0.
I20260808 07:13:45.246289 5187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7fa7f0210c134378b07590568096bc75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67f1477558ab4184be2dd31ae3c950dd" is_pre_election: true
I20260808 07:13:45.246423 5187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fa7f0210c134378b07590568096bc75 in term 0.
I20260808 07:13:45.246809 5243 leader_election.cc:302] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67f1477558ab4184be2dd31ae3c950dd, 7fa7f0210c134378b07590568096bc75; no voters:
I20260808 07:13:45.247045 5327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:45.247100 5327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:45.247123 5327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:45.248193 5327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.248309 5327 leader_election.cc:288] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [CANDIDATE]: Term 1 election: Requested vote from peers be451b335b634c8db60f99754f557315 (127.22.2.190:40623), 67f1477558ab4184be2dd31ae3c950dd (127.22.2.189:44335)
I20260808 07:13:45.248611 5100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7fa7f0210c134378b07590568096bc75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be451b335b634c8db60f99754f557315"
I20260808 07:13:45.248687 5100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:45.249500 5100 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fa7f0210c134378b07590568096bc75 in term 1.
I20260808 07:13:45.249701 5187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7fa7f0210c134378b07590568096bc75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67f1477558ab4184be2dd31ae3c950dd"
I20260808 07:13:45.249774 5187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:45.250617 5187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fa7f0210c134378b07590568096bc75 in term 1.
I20260808 07:13:45.250909 5243 leader_election.cc:302] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67f1477558ab4184be2dd31ae3c950dd, 7fa7f0210c134378b07590568096bc75; no voters:
I20260808 07:13:45.251892 5327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:45.252067 5327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa7f0210c134378b07590568096bc75, State: Running, Role: LEADER
I20260808 07:13:45.252149 5327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } }
I20260808 07:13:45.252516 5327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fa7f0210c134378b07590568096bc75. Latest consensus state: current_term: 1 leader_uuid: "7fa7f0210c134378b07590568096bc75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.252578 5327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:45.252874 5367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:45.253077 5367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:45.255076 5187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [term 1 FOLLOWER]: Refusing update from remote peer 7fa7f0210c134378b07590568096bc75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:45.255337 5100 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [term 1 FOLLOWER]: Refusing update from remote peer 7fa7f0210c134378b07590568096bc75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:45.255739 5327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:45.256117 5327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:45.257395 5312 sys_catalog.cc:455] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fa7f0210c134378b07590568096bc75. Latest consensus state: current_term: 1 leader_uuid: "7fa7f0210c134378b07590568096bc75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.257481 5312 sys_catalog.cc:458] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:45.258652 5351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fa7f0210c134378b07590568096bc75. Latest consensus state: current_term: 1 leader_uuid: "7fa7f0210c134378b07590568096bc75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.258739 5351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:45.260103 5327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7fa7f0210c134378b07590568096bc75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.260190 5327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:45.260581 5312 sys_catalog.cc:455] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7fa7f0210c134378b07590568096bc75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.260655 5312 sys_catalog.cc:458] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:45.260941 5368 mvcc.cc:205] Tried to move back new op lower bound from 7316165530642280448 to 7316165530633461760. Current Snapshot: MvccSnapshot[applied={T|T < 7316165530642280448}]
I20260808 07:13:45.261583 5367 catalog_manager.cc:1347] Generated new cluster ID: f119327f46e241858d31b5c08bb94ce0
I20260808 07:13:45.261626 5367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:45.270684 5365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7fa7f0210c134378b07590568096bc75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.270782 5365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:45.271504 5351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7fa7f0210c134378b07590568096bc75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be451b335b634c8db60f99754f557315" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40623 } } peers { permanent_uuid: "67f1477558ab4184be2dd31ae3c950dd" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 44335 } } peers { permanent_uuid: "7fa7f0210c134378b07590568096bc75" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 35077 } } }
I20260808 07:13:45.271730 5351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:45.271850 5371 mvcc.cc:205] Tried to move back new op lower bound from 7316165530642280448 to 7316165530633461760. Current Snapshot: MvccSnapshot[applied={T|T < 7316165530642280448}]
I20260808 07:13:45.271957 5369 mvcc.cc:205] Tried to move back new op lower bound from 7316165530642280448 to 7316165530633461760. Current Snapshot: MvccSnapshot[applied={T|T < 7316165530642280448}]
I20260808 07:13:45.273676 5367 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:45.274420 5367 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:45.286154 5367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fa7f0210c134378b07590568096bc75: Generated new TSK 0
I20260808 07:13:45.286657 5367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:13:46.152701 5324 catalog_manager.cc:1259] Loaded cluster ID: f119327f46e241858d31b5c08bb94ce0
I20260808 07:13:46.152777 5324 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315: loading cluster ID for follower catalog manager: success
I20260808 07:13:46.153609 5324 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315: acquiring CA information for follower catalog manager: success
I20260808 07:13:46.154083 5324 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be451b335b634c8db60f99754f557315: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:13:46.201452 5363 catalog_manager.cc:1259] Loaded cluster ID: f119327f46e241858d31b5c08bb94ce0
I20260808 07:13:46.201644 5363 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd: loading cluster ID for follower catalog manager: success
I20260808 07:13:46.202468 5363 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd: acquiring CA information for follower catalog manager: success
I20260808 07:13:46.202772 5363 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 67f1477558ab4184be2dd31ae3c950dd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:13:46.343148 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 5048
I20260808 07:13:46.350626 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 5136
I20260808 07:13:46.356858 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 5222
2026-08-08T07:13:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2933 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:13:46 dist-test-slave-6k22 krb5kdc[5386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:13:46 dist-test-slave-6k22 krb5kdc[5386](info): set up 2 sockets
Aug 08 07:13:46 dist-test-slave-6k22 krb5kdc[5386](info): commencing operation
krb5kdc: starting...
W20260808 07:13:47.475173 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:13:47 dist-test-slave-6k22 krb5kdc[5386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:13:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:13:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:13:47.596423 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40703
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:40489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:40703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:47.679011 5401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:47.679267 5401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:47.679307 5401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:47.680454 5401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:47.680527 5401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:47.680552 5401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:47.680572 5401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:47.680601 5401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:47.682080 5401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:40489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:40703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:40703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:47.682358 5401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:47.682582 5401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:47.687589 5409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:47.687593 5410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:47.687736 5401 server_base.cc:1034] running on GCE node
W20260808 07:13:47.687599 5412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:47.688081 5401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:47.688390 5401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:47.689561 5401 hybrid_clock.cc:648] HybridClock initialized: now 1786173227689546 us; error 44 us; skew 500 ppm
Aug 08 07:13:47 dist-test-slave-6k22 krb5kdc[5386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173227, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:47.692170 5401 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:13:47.692622 5401 webserver.cc:460] Webserver started at http://127.22.2.190:37673/ using document root <none> and password file <none>
I20260808 07:13:47.692803 5401 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:47.692843 5401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:47.692929 5401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:47.693923 5401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "36e7b375df374ec29dbdfcbb28457182"
format_stamp: "Formatted at 2026-08-08 07:13:47 on dist-test-slave-6k22"
server_key: "46a467fe4c59ae30a8083f87a5a73acf"
server_key_iv: "7909ed2f193817a9bde6ebe0cea56857"
server_key_version: "encryptionkey@0"
I20260808 07:13:47.694223 5401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "36e7b375df374ec29dbdfcbb28457182"
format_stamp: "Formatted at 2026-08-08 07:13:47 on dist-test-slave-6k22"
server_key: "46a467fe4c59ae30a8083f87a5a73acf"
server_key_iv: "7909ed2f193817a9bde6ebe0cea56857"
server_key_version: "encryptionkey@0"
I20260808 07:13:47.695463 5401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:13:47.696234 5419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:47.696434 5401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:47.696527 5401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "36e7b375df374ec29dbdfcbb28457182"
format_stamp: "Formatted at 2026-08-08 07:13:47 on dist-test-slave-6k22"
server_key: "46a467fe4c59ae30a8083f87a5a73acf"
server_key_iv: "7909ed2f193817a9bde6ebe0cea56857"
server_key_version: "encryptionkey@0"
I20260808 07:13:47.696585 5401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:47.729868 5401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:47.732348 5401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:47.732478 5401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:47.736418 5401 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:40703
I20260808 07:13:47.736505 5479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:40703 every 8 connection(s)
I20260808 07:13:47.736853 5401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:47.737766 5480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:47.740118 5480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182: Bootstrap starting.
I20260808 07:13:47.740715 5480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:47.741021 5480 log.cc:824] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:47.741597 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 5401
I20260808 07:13:47.741721 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:13:47.741827 5480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182: No bootstrap required, opened a new log
I20260808 07:13:47.741942 22538 external_mini_cluster.cc:1428] Setting key 6c8e4dd46673841a822215ad8f8d10e5
I20260808 07:13:47.744275 5480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e7b375df374ec29dbdfcbb28457182" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40703 } }
I20260808 07:13:47.744458 5480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:47.744508 5480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36e7b375df374ec29dbdfcbb28457182, State: Initialized, Role: FOLLOWER
I20260808 07:13:47.744640 5480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e7b375df374ec29dbdfcbb28457182" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40703 } }
I20260808 07:13:47.744700 5480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:47.744722 5480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:47.744773 5480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:47.745312 5480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e7b375df374ec29dbdfcbb28457182" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40703 } }
I20260808 07:13:47.745397 5480 leader_election.cc:302] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36e7b375df374ec29dbdfcbb28457182; no voters:
I20260808 07:13:47.745608 5480 leader_election.cc:288] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:13:47.745628 5486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:47.745756 5486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [term 1 LEADER]: Becoming Leader. State: Replica: 36e7b375df374ec29dbdfcbb28457182, State: Running, Role: LEADER
I20260808 07:13:47.745893 5480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:47.745918 5486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e7b375df374ec29dbdfcbb28457182" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40703 } }
I20260808 07:13:47.746467 5487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36e7b375df374ec29dbdfcbb28457182. Latest consensus state: current_term: 1 leader_uuid: "36e7b375df374ec29dbdfcbb28457182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e7b375df374ec29dbdfcbb28457182" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40703 } } }
I20260808 07:13:47.746560 5487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:47.746788 5488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36e7b375df374ec29dbdfcbb28457182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e7b375df374ec29dbdfcbb28457182" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 40703 } } }
I20260808 07:13:47.747068 5488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:47.747306 5493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:47.747872 5493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:47.749585 5493 catalog_manager.cc:1347] Generated new cluster ID: 5b5ac362334c4a9f8a2c33e8b267d37a
I20260808 07:13:47.749635 5493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:13:47 dist-test-slave-6k22 krb5kdc[5386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:47.754436 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:47.742445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35890 (local address 127.22.2.190:40703)
0808 07:13:47.742624 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:47.742631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:47.742657 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:13:47.743199 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:47.743207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:47.743252 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:47.743667 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:47.744456 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:47.745863 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:47.746850 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:47.747504 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:47.748022 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:47.750540 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:47.750544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:47.750544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:47.750556 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:47.752610 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:47.753024 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:47.753028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:47.753028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:47.753138 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:47.753342 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:47.753343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:47.753344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:47.753668 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:47.753847 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:47.753923 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:13:47.754093 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260808 07:13:47.755533 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:13:47.776342 5493 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:47.776957 5493 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:47.780248 5493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36e7b375df374ec29dbdfcbb28457182: Generated new TSK 0
I20260808 07:13:47.780435 5493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:13:47.809718 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 5401
2026-08-08T07:13:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1451 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:13:47 dist-test-slave-6k22 krb5kdc[5511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:13:47 dist-test-slave-6k22 krb5kdc[5511](info): set up 2 sockets
Aug 08 07:13:47 dist-test-slave-6k22 krb5kdc[5511](info): commencing operation
krb5kdc: starting...
W20260808 07:13:48.942215 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:13:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:13:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:13:49.068631 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44035
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:49.151732 5527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:49.151932 5527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:49.151955 5527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:49.153106 5527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:49.153149 5527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:49.153163 5527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:49.153175 5527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:49.153187 5527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:49.154719 5527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:44035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:49.154994 5527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:49.155243 5527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:49.160274 5534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:49.160316 5537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:49.160291 5535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:49.160554 5527 server_base.cc:1034] running on GCE node
I20260808 07:13:49.160907 5527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:49.161237 5527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:49.162400 5527 hybrid_clock.cc:648] HybridClock initialized: now 1786173229162377 us; error 40 us; skew 500 ppm
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:49.169576 5527 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:13:49.170053 5527 webserver.cc:460] Webserver started at http://127.22.2.190:42659/ using document root <none> and password file <none>
I20260808 07:13:49.170305 5527 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:49.170349 5527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:49.170514 5527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:49.171746 5527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "aa31b50a1df444d0bd0ed440b5fea976"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "f07a9fab1f0da7455d859bd2f096cc5c"
server_key_iv: "deec98180a62a2f8aecdd4ce16659307"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.172116 5527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "aa31b50a1df444d0bd0ed440b5fea976"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "f07a9fab1f0da7455d859bd2f096cc5c"
server_key_iv: "deec98180a62a2f8aecdd4ce16659307"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.173580 5527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:13:49.174371 5544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.174561 5527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:13:49.174711 5527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "aa31b50a1df444d0bd0ed440b5fea976"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "f07a9fab1f0da7455d859bd2f096cc5c"
server_key_iv: "deec98180a62a2f8aecdd4ce16659307"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.174775 5527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:49.188764 5527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:49.190598 5527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:49.190769 5527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:49.195164 5527 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:44035
I20260808 07:13:49.195199 5601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:44035 every 8 connection(s)
I20260808 07:13:49.195545 5527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:49.196612 5602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:49.198916 5602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Bootstrap starting.
I20260808 07:13:49.199503 5602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:49.199828 5602 log.cc:824] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:49.200505 5602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: No bootstrap required, opened a new log
I20260808 07:13:49.202044 5602 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:49.202176 5602 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:49.202198 5602 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa31b50a1df444d0bd0ed440b5fea976, State: Initialized, Role: FOLLOWER
I20260808 07:13:49.202339 5602 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:49.202421 5602 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:49.202445 5602 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:49.202472 5602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:49.203217 5602 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:49.203301 5602 leader_election.cc:302] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa31b50a1df444d0bd0ed440b5fea976; no voters:
I20260808 07:13:49.203428 5602 leader_election.cc:288] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:13:49.203493 5606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:49.203704 5602 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:49.203770 5606 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 1 LEADER]: Becoming Leader. State: Replica: aa31b50a1df444d0bd0ed440b5fea976, State: Running, Role: LEADER
I20260808 07:13:49.203893 5606 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:49.204020 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 5527
I20260808 07:13:49.204144 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:13:49.204370 22538 external_mini_cluster.cc:1428] Setting key da50b58135278d6f77afb1f8dabce676
I20260808 07:13:49.204377 5607 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa31b50a1df444d0bd0ed440b5fea976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } } }
I20260808 07:13:49.204368 5608 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa31b50a1df444d0bd0ed440b5fea976. Latest consensus state: current_term: 1 leader_uuid: "aa31b50a1df444d0bd0ed440b5fea976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } } }
I20260808 07:13:49.204466 5607 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:49.204473 5608 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:49.204682 5618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:49.205289 5618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:49.207837 5618 catalog_manager.cc:1347] Generated new cluster ID: dbd9bafd16d24b67ae12d08ec6214c44
I20260808 07:13:49.207921 5618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:49.216729 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.205060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.22.2.190:44035)
0808 07:13:49.205745 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:49.205754 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:49.205806 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:13:49.205898 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:49.205904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:49.205945 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:49.206087 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:49.206907 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.208254 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.209368 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.210007 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.210108 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.212953 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:49.212958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:49.212958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:49.212973 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:49.215023 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.215423 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.215426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.215427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.215586 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.215793 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.215794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.215794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.216099 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:49.216253 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:49.216356 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:13:49.216498 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":63,"threads_started":1}
I20260808 07:13:49.233304 5618 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:49.233893 5618 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:49.239773 5618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Generated new TSK 0
I20260808 07:13:49.239946 5618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:13:49.270833 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:44035
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:49.353940 5633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:49.354136 5633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:49.354158 5633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:49.355299 5633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:49.355341 5633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:49.355367 5633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:13:49.356890 5633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:49.357116 5633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:49.357386 5633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:49.362581 5638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:49.362582 5639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:49.362572 5641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:49.362646 5633 server_base.cc:1034] running on GCE node
I20260808 07:13:49.363071 5633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:49.363340 5633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:49.364514 5633 hybrid_clock.cc:648] HybridClock initialized: now 1786173229364507 us; error 36 us; skew 500 ppm
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:49.367197 5633 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:13:49.367738 5633 webserver.cc:460] Webserver started at http://127.22.2.129:43967/ using document root <none> and password file <none>
I20260808 07:13:49.367926 5633 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:49.367965 5633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:49.368054 5633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:49.369055 5633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "83c908b58dde47e1b89e9faeab6b3cf3"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "e85809b0b9f808fb915fec374f080277"
server_key_iv: "a1ce1ea2ad0ee109eed7af3834328028"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.369360 5633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "83c908b58dde47e1b89e9faeab6b3cf3"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "e85809b0b9f808fb915fec374f080277"
server_key_iv: "a1ce1ea2ad0ee109eed7af3834328028"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.370594 5633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:13:49.371301 5650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.371462 5633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:13:49.371591 5633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "83c908b58dde47e1b89e9faeab6b3cf3"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "e85809b0b9f808fb915fec374f080277"
server_key_iv: "a1ce1ea2ad0ee109eed7af3834328028"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.371671 5633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:49.391688 5633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:49.393664 5633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:49.393806 5633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:49.394121 5633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:49.394585 5633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:49.394635 5633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.394670 5633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:49.394691 5633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.400592 5633 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:37145
I20260808 07:13:49.400730 5776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:37145 every 8 connection(s)
I20260808 07:13:49.401015 5633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:49.406435 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 5633
I20260808 07:13:49.406530 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:13:49.406795 22538 external_mini_cluster.cc:1428] Setting key c272239a93d222d1bb75c61d6522285d
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:49.411137 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.401806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:37147)
0808 07:13:49.401998 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:49.402034 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:49.402512 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:49.402769 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:49.402772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:49.403343 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:49.403532 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.403536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.404404 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.404405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.405338 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.405340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.405743 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.405744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.405877 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.406349 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:49.406357 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:49.408176 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:49.409660 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:49.409667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:49.409670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:49.409997 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:49.410190 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:49.410191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:49.410191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:49.410333 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:49.410553 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:49.410560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:49.410723 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 07:13:49.410858 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 07:13:49.410975 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":114,"threads_started":1}
I20260808 07:13:49.411142 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.401837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:37147 (local address 127.22.2.190:44035)
0808 07:13:49.401916 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:49.401918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:49.402124 (+ 206us) server_negotiation.cc:407] Connection header received
0808 07:13:49.402633 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:49.402634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:49.402652 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:49.402720 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:49.403679 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.404300 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.405427 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.405645 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.405729 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.408279 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:49.408280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:49.408280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:49.408286 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:49.409595 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.410072 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.410073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.410073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.410123 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.410396 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.410396 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.410396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.410491 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:49.410540 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:49.410909 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 07:13:49.410999 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:13:49.411830 5778 heartbeater.cc:347] Connected to a master server at 127.22.2.190:44035
I20260808 07:13:49.411937 5778 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:49.412106 5778 heartbeater.cc:511] Master 127.22.2.190:44035 requested a full tablet report, sending...
I20260808 07:13:49.412755 5563 ts_manager.cc:194] Registered new tserver with Master: 83c908b58dde47e1b89e9faeab6b3cf3 (127.22.2.129:37145)
I20260808 07:13:49.413751 5563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:37147
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:13:49.435452 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:44035
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:49.521101 5784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:49.521323 5784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:49.521361 5784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:49.522486 5784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:49.522548 5784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:49.522591 5784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:13:49.524044 5784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:49.524309 5784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:49.524549 5784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:49.529527 5789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:49.529533 5790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:49.529572 5792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:49.529743 5784 server_base.cc:1034] running on GCE node
I20260808 07:13:49.529964 5784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:49.530198 5784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:49.531356 5784 hybrid_clock.cc:648] HybridClock initialized: now 1786173229531322 us; error 54 us; skew 500 ppm
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:49.533906 5784 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:13:49.534368 5784 webserver.cc:460] Webserver started at http://127.22.2.130:46467/ using document root <none> and password file <none>
I20260808 07:13:49.534607 5784 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:49.534674 5784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:49.534833 5784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:49.535921 5784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "6c018e1ea4374710a59a5d792605a101"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "7dca1e6b606ef8866d46be9e9c09a75e"
server_key_iv: "f9d90caeb2afd8454b10e4ccaa17ccd0"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.536273 5784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "6c018e1ea4374710a59a5d792605a101"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "7dca1e6b606ef8866d46be9e9c09a75e"
server_key_iv: "f9d90caeb2afd8454b10e4ccaa17ccd0"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.537659 5784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:13:49.538370 5801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.538554 5784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.538651 5784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "6c018e1ea4374710a59a5d792605a101"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "7dca1e6b606ef8866d46be9e9c09a75e"
server_key_iv: "f9d90caeb2afd8454b10e4ccaa17ccd0"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.538731 5784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:49.562089 5784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:49.563997 5784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:49.564158 5784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:49.564396 5784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:49.564723 5784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:49.564778 5784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.564850 5784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:49.564881 5784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.571322 5784 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35017
I20260808 07:13:49.571377 5936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35017 every 8 connection(s)
I20260808 07:13:49.571753 5784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:49.581379 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 5784
I20260808 07:13:49.581501 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:13:49.581758 22538 external_mini_cluster.cc:1428] Setting key 57e034414a44d2ac476c94b4b6238d74
I20260808 07:13:49.582425 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.572540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:50559 (local address 127.22.2.190:44035)
0808 07:13:49.572625 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:49.572627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:49.572869 (+ 242us) server_negotiation.cc:407] Connection header received
0808 07:13:49.573447 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:49.573449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:49.573463 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:49.573527 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:49.574701 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.575311 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.576498 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.576745 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.576831 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.579185 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:49.579186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:49.579187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:49.579194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:49.580866 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.581241 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.581242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.581243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.581318 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.581607 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.581608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.581609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.581820 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:49.581900 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:49.582253 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 07:13:49.582326 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:13:49.582661 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.572506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:50559)
0808 07:13:49.572734 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:49.572773 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:49.573331 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:49.573609 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:49.573613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:49.574374 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:49.574585 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.574590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.575429 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.575430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.576384 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.576386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.576843 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.576844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.576983 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.577586 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:49.577596 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:49.579088 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:49.580938 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:49.580944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:49.580945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:49.581165 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:49.581395 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:49.581395 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:49.581396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:49.581517 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:49.581912 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:49.581918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:49.582089 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 07:13:49.582227 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 07:13:49.582397 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":83,"threads_started":1}
I20260808 07:13:49.583163 5937 heartbeater.cc:347] Connected to a master server at 127.22.2.190:44035
I20260808 07:13:49.583250 5937 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:49.583444 5937 heartbeater.cc:511] Master 127.22.2.190:44035 requested a full tablet report, sending...
I20260808 07:13:49.583849 5563 ts_manager.cc:194] Registered new tserver with Master: 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
I20260808 07:13:49.584359 5563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:50559
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:13:49.608220 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:44035
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:49.691161 5943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:49.691447 5943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:49.691490 5943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:49.692788 5943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:49.692855 5943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:49.692903 5943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:13:49.694409 5943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:49.694690 5943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:49.694963 5943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:49.700395 5949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:49.700480 5943 server_base.cc:1034] running on GCE node
W20260808 07:13:49.700394 5952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:49.700544 5950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:49.700798 5943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:49.701062 5943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:49.702219 5943 hybrid_clock.cc:648] HybridClock initialized: now 1786173229702214 us; error 39 us; skew 500 ppm
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:49.704993 5943 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:13:49.705366 5943 webserver.cc:460] Webserver started at http://127.22.2.131:44551/ using document root <none> and password file <none>
I20260808 07:13:49.705598 5943 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:49.705665 5943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:49.705802 5943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:49.706833 5943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "1517e22aabb6497d81322ee50b34c789"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "31fc213f1a4e6f420150e95b0a0632b0"
server_key_iv: "6775f47b3450ea7bb86f07f4a71a4b19"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.707131 5943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "1517e22aabb6497d81322ee50b34c789"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "31fc213f1a4e6f420150e95b0a0632b0"
server_key_iv: "6775f47b3450ea7bb86f07f4a71a4b19"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.708465 5943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:13:49.709195 5960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.709381 5943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:49.709482 5943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "1517e22aabb6497d81322ee50b34c789"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "31fc213f1a4e6f420150e95b0a0632b0"
server_key_iv: "6775f47b3450ea7bb86f07f4a71a4b19"
server_key_version: "encryptionkey@0"
I20260808 07:13:49.709565 5943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:49.718983 5943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:49.721312 5943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:49.721475 5943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:49.721699 5943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:49.722007 5943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:49.722061 5943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.722102 5943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:49.722126 5943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:49.728042 5943 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:40803
I20260808 07:13:49.728117 6082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:40803 every 8 connection(s)
I20260808 07:13:49.728425 5943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:13:49.733093 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 5943
I20260808 07:13:49.733199 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:13:49.733412 22538 external_mini_cluster.cc:1428] Setting key 1bd60b15306445682b7ac371202c189a
Aug 08 07:13:49 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173229, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:49.738258 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.729130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:50791 (local address 127.22.2.190:44035)
0808 07:13:49.729218 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:49.729221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:49.729402 (+ 181us) server_negotiation.cc:407] Connection header received
0808 07:13:49.729950 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:49.729953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:49.729965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:49.730040 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:49.730976 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.731485 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.732551 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.732773 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.732860 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.735341 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:49.735342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:49.735343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:49.735352 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:49.736790 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.737271 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.737272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.737273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.737339 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.737603 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.737604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.737605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.737732 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:49.737811 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:49.738086 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 07:13:49.738167 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:13:49.738477 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.729085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:50791)
0808 07:13:49.729277 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:49.729321 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:49.729821 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:49.730129 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:49.730133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:49.730688 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:49.730877 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.730881 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.731608 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.731609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.732426 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.732428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.732873 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.732873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.733004 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.733547 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:49.733557 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:49.735244 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:49.736895 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:49.736903 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:49.736905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:49.737171 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:49.737415 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:49.737416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:49.737416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:49.737534 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:49.737801 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:49.737808 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:49.737948 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:13:49.738052 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:13:49.738210 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
I20260808 07:13:49.739140 6084 heartbeater.cc:347] Connected to a master server at 127.22.2.190:44035
I20260808 07:13:49.739249 6084 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:49.739435 6084 heartbeater.cc:511] Master 127.22.2.190:44035 requested a full tablet report, sending...
I20260808 07:13:49.739866 5563 ts_manager.cc:194] Registered new tserver with Master: 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803)
I20260808 07:13:49.740255 5563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:50791
I20260808 07:13:49.745177 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:49.752146 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.746492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.22.2.190:44035)
0808 07:13:49.746555 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:49.746556 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:49.746642 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:13:49.746689 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:49.746690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:49.746698 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:49.746737 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:49.747261 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.747787 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.748476 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.748649 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.748697 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.749832 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:49.749833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:49.749833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:49.749839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:49.751178 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.751462 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.751463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.751463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.751512 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:49.751711 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:49.751711 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:49.751712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:49.751845 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:49.751918 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:49.752008 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 07:13:49.752062 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:13:49.753583 5563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:13:49.755167 5563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:13:49.761996 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.757475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40803 (local address 127.0.0.1:41596)
0808 07:13:49.757748 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:49.757791 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:49.757888 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:49.758141 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:49.758145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:49.758148 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:49.758213 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.758215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.759602 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.759603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.760751 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.760754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.761630 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.761631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.761716 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.761743 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 07:13:49.761811 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:13:49.761868 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":76,"thread_start_us":100,"threads_started":1}
I20260808 07:13:49.762115 6103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.757740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41596 (local address 127.22.2.131:40803)
0808 07:13:49.757961 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:49.757966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:49.757978 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:13:49.758020 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:49.758022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:49.758040 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:49.758125 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:49.758309 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.759464 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.760851 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.761536 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.761612 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.761668 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:49.761980 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 07:13:49.762032 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":110,"threads_started":1}
I20260808 07:13:49.762166 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.757612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.22.2.130:35017)
0808 07:13:49.758263 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:49.758268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:49.758277 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:49.758318 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:49.758320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:49.758335 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:49.758422 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:49.758647 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.759548 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.761131 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.761671 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.761737 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.761791 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:49.762033 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:13:49.762089 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":121,"threads_started":1}
I20260808 07:13:49.762346 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.757759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40622 (local address 127.22.2.129:37145)
0808 07:13:49.757972 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:49.757976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:49.758137 (+ 161us) server_negotiation.cc:407] Connection header received
0808 07:13:49.758185 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:49.758187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:49.758203 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:49.758280 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:49.758528 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.759597 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.761116 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.761818 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.761886 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.761935 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:49.762189 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 07:13:49.762261 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
I20260808 07:13:49.762115 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.757477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37145 (local address 127.0.0.1:40622)
0808 07:13:49.758024 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:49.758035 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:49.758103 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:49.758393 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:49.758393 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:49.758395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:49.758443 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.758445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.759704 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.759705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.761030 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.761033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.761893 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.761894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.761960 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.761964 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:49.762017 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:13:49.762054 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":43,"thread_start_us":96,"threads_started":1}
I20260808 07:13:49.762640 6099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:49.757477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35017 (local address 127.0.0.1:48384)
0808 07:13:49.757840 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:49.757849 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:49.757892 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:49.758516 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:49.758517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:49.758518 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:49.758560 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.758561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.759739 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.759740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.761030 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:49.761033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:49.761753 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:49.761754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:49.761817 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:49.761821 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:49.761883 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:13:49.761927 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":76,"thread_start_us":57,"threads_started":1}
I20260808 07:13:49.763165 6012 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced (DEFAULT_TABLE table=test-table [id=1056f4e99e904b90bcded0b96837f131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:49.763522 6012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:49.763408 5704 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced (DEFAULT_TABLE table=test-table [id=1056f4e99e904b90bcded0b96837f131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:49.763468 5862 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced (DEFAULT_TABLE table=test-table [id=1056f4e99e904b90bcded0b96837f131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:49.763767 5704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:49.763765 5862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:49.766104 6105 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Bootstrap starting.
I20260808 07:13:49.766742 6105 tablet_bootstrap.cc:654] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:49.767364 6105 log.cc:824] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:49.767722 6107 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Bootstrap starting.
I20260808 07:13:49.768244 6105 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: No bootstrap required, opened a new log
I20260808 07:13:49.768321 6105 ts_tablet_manager.cc:1397] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 07:13:49.768385 6107 tablet_bootstrap.cc:654] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:49.768935 6107 log.cc:824] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:49.769316 6106 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Bootstrap starting.
I20260808 07:13:49.769634 6107 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: No bootstrap required, opened a new log
I20260808 07:13:49.769727 6107 ts_tablet_manager.cc:1397] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:13:49.769987 6106 tablet_bootstrap.cc:654] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:49.770538 6106 log.cc:824] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:49.770591 6105 raft_consensus.cc:348] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:49.770859 6105 raft_consensus.cc:374] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:49.770927 6105 raft_consensus.cc:729] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1517e22aabb6497d81322ee50b34c789, State: Initialized, Role: FOLLOWER
I20260808 07:13:49.771101 6105 consensus_queue.cc:260] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:49.771322 6106 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: No bootstrap required, opened a new log
I20260808 07:13:49.771389 6106 ts_tablet_manager.cc:1397] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:13:49.771406 6105 ts_tablet_manager.cc:1428] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:13:49.771530 6084 heartbeater.cc:503] Master 127.22.2.190:44035 was elected leader, sending a full tablet report...
I20260808 07:13:49.771651 6107 raft_consensus.cc:348] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:49.771790 6107 raft_consensus.cc:374] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:49.771824 6107 raft_consensus.cc:729] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c018e1ea4374710a59a5d792605a101, State: Initialized, Role: FOLLOWER
I20260808 07:13:49.771932 6107 consensus_queue.cc:260] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:49.772243 6107 ts_tablet_manager.cc:1428] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:13:49.772295 5937 heartbeater.cc:503] Master 127.22.2.190:44035 was elected leader, sending a full tablet report...
I20260808 07:13:49.773259 6106 raft_consensus.cc:348] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:49.773427 6106 raft_consensus.cc:374] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:49.773473 6106 raft_consensus.cc:729] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c908b58dde47e1b89e9faeab6b3cf3, State: Initialized, Role: FOLLOWER
I20260808 07:13:49.773578 6106 consensus_queue.cc:260] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:49.773841 6106 ts_tablet_manager.cc:1428] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:13:49.773909 5778 heartbeater.cc:503] Master 127.22.2.190:44035 was elected leader, sending a full tablet report...
W20260808 07:13:49.822366 5939 tablet.cc:2369] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:13:49.901875 5780 tablet.cc:2369] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:13:49.978997 6085 tablet.cc:2369] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:50.044497 6113 raft_consensus.cc:482] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:50.044708 6113 raft_consensus.cc:504] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:50.045109 6113 leader_election.cc:288] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017), 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803)
I20260808 07:13:50.048441 6103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.045415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:42785 (local address 127.22.2.131:40803)
0808 07:13:50.045458 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.045460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.045558 (+ 98us) server_negotiation.cc:407] Connection header received
0808 07:13:50.045640 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.045641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.045671 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.045730 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:50.045994 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.046468 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.047675 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.048077 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.048134 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.048141 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.048316 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.048366 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:13:50.048444 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.045311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40803 (local address 127.22.2.129:42785)
0808 07:13:50.045468 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.045479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.045571 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.045748 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.045749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.045751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:50.045898 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.045900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.046564 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.046564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.047573 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.047576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.048153 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.048154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.048235 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.048240 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.048293 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.048335 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":3,"thread_start_us":66,"threads_started":1}
I20260808 07:13:50.048538 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.045354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:59907 (local address 127.22.2.130:35017)
0808 07:13:50.045416 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.045418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.045502 (+ 84us) server_negotiation.cc:407] Connection header received
0808 07:13:50.045599 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.045600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.045615 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.045676 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:50.045901 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.046417 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.047717 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.048158 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.048230 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.048237 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.048421 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.048465 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:13:50.048789 6115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.045262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35017 (local address 127.22.2.129:59907)
0808 07:13:50.045432 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.045444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.045508 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.045695 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.045736 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.045739 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:50.045821 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.045824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.046497 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.046498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.047598 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.047602 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.048248 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.048249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.048338 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.048341 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.048398 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.048438 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":5,"thread_start_us":70,"threads_started":1}
I20260808 07:13:50.048887 6033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe5f15b8a6a4ebb9ce36f546d22cced" candidate_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1517e22aabb6497d81322ee50b34c789" is_pre_election: true
I20260808 07:13:50.049036 6033 raft_consensus.cc:2454] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83c908b58dde47e1b89e9faeab6b3cf3 in term 0.
I20260808 07:13:50.049093 6111 raft_consensus.cc:482] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:50.049139 6111 raft_consensus.cc:504] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:50.049127 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe5f15b8a6a4ebb9ce36f546d22cced" candidate_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c018e1ea4374710a59a5d792605a101" is_pre_election: true
I20260808 07:13:50.049268 5651 leader_election.cc:302] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1517e22aabb6497d81322ee50b34c789, 83c908b58dde47e1b89e9faeab6b3cf3; no voters:
I20260808 07:13:50.049299 5883 raft_consensus.cc:2454] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83c908b58dde47e1b89e9faeab6b3cf3 in term 0.
I20260808 07:13:50.049408 6113 raft_consensus.cc:2781] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:50.049448 6111 leader_election.cc:288] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83c908b58dde47e1b89e9faeab6b3cf3 (127.22.2.129:37145), 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
I20260808 07:13:50.049474 6113 raft_consensus.cc:482] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:50.049500 6113 raft_consensus.cc:3037] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:50.050540 6113 raft_consensus.cc:504] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:50.050668 6113 leader_election.cc:288] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 election: Requested vote from peers 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017), 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803)
I20260808 07:13:50.050804 6033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe5f15b8a6a4ebb9ce36f546d22cced" candidate_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1517e22aabb6497d81322ee50b34c789"
I20260808 07:13:50.050859 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe5f15b8a6a4ebb9ce36f546d22cced" candidate_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c018e1ea4374710a59a5d792605a101"
I20260808 07:13:50.050880 6033 raft_consensus.cc:3037] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:50.050935 5883 raft_consensus.cc:3037] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:50.051826 5883 raft_consensus.cc:2454] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c908b58dde47e1b89e9faeab6b3cf3 in term 1.
I20260808 07:13:50.051863 6033 raft_consensus.cc:2454] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c908b58dde47e1b89e9faeab6b3cf3 in term 1.
I20260808 07:13:50.051997 5654 leader_election.cc:302] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c018e1ea4374710a59a5d792605a101, 83c908b58dde47e1b89e9faeab6b3cf3; no voters:
I20260808 07:13:50.052208 6113 raft_consensus.cc:2781] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:50.052361 6113 raft_consensus.cc:686] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 LEADER]: Becoming Leader. State: Replica: 83c908b58dde47e1b89e9faeab6b3cf3, State: Running, Role: LEADER
I20260808 07:13:50.052444 6113 consensus_queue.cc:237] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:50.052865 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.049568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37145 (local address 127.22.2.131:48497)
0808 07:13:50.049652 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.049664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.049703 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.049976 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.049978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.049979 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:50.050034 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.050035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.050629 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.050630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.051772 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.051775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.052576 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.052577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.052711 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.052714 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.052771 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.052807 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 07:13:50.053026 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.049655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:39101 (local address 127.22.2.130:35017)
0808 07:13:50.049690 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.049691 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.049858 (+ 167us) server_negotiation.cc:407] Connection header received
0808 07:13:50.049954 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.049955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.049962 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.049990 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:50.050183 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.050809 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.052125 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.052633 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.052679 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.052685 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.052924 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.052970 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:13:50.053207 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.049771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:48497 (local address 127.22.2.129:37145)
0808 07:13:50.049806 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.049808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.049815 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:13:50.049837 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.049838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.049847 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.049931 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:50.050097 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.050541 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.051874 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.052338 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.052433 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.052441 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.053094 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.053132 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:13:50.053251 6117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.049598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35017 (local address 127.22.2.131:39101)
0808 07:13:50.049794 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.049806 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.049885 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.050045 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.050046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.050047 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:50.050112 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.050114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.050903 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.050904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.052028 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.052031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.052752 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.052753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.052839 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.052843 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.053139 (+ 296us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.053179 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":121,"threads_started":1}
I20260808 07:13:50.053225 5563 catalog_manager.cc:5495] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83c908b58dde47e1b89e9faeab6b3cf3 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } health_report { overall_health: UNKNOWN } } }
I20260808 07:13:50.053418 5724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe5f15b8a6a4ebb9ce36f546d22cced" candidate_uuid: "1517e22aabb6497d81322ee50b34c789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" is_pre_election: true
I20260808 07:13:50.053517 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe5f15b8a6a4ebb9ce36f546d22cced" candidate_uuid: "1517e22aabb6497d81322ee50b34c789" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c018e1ea4374710a59a5d792605a101" is_pre_election: true
I20260808 07:13:50.053596 5883 raft_consensus.cc:2379] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1517e22aabb6497d81322ee50b34c789 in current term 1: Already voted for candidate 83c908b58dde47e1b89e9faeab6b3cf3 in this term.
I20260808 07:13:50.053853 5965 leader_election.cc:302] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1517e22aabb6497d81322ee50b34c789; no voters: 6c018e1ea4374710a59a5d792605a101, 83c908b58dde47e1b89e9faeab6b3cf3
I20260808 07:13:50.053984 6111 raft_consensus.cc:2726] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 07:13:50.115625 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.112877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40634 (local address 127.22.2.129:37145)
0808 07:13:50.112940 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.112943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.112953 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:50.113009 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.113010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.113031 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.113144 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:50.113324 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.113851 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.114732 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.114949 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.115044 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.115231 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:50.115344 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:50.115446 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.115475 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.115497 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:13:50.117515 5883 raft_consensus.cc:1261] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Refusing update from remote peer 83c908b58dde47e1b89e9faeab6b3cf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:50.117527 6033 raft_consensus.cc:1261] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Refusing update from remote peer 83c908b58dde47e1b89e9faeab6b3cf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:50.117877 6113 consensus_queue.cc:1035] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:50.118011 6118 consensus_queue.cc:1035] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:50.121184 6122 mvcc.cc:205] Tried to move back new op lower bound from 7316165550558617600 to 7316165550295306240. Current Snapshot: MvccSnapshot[applied={T|T < 7316165550558617600 or (T in {7316165550558617600})}]
I20260808 07:13:50.122942 5562 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:50.123059 5562 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:50.123919 5562 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Sending DeleteTablet for 3 replicas of tablet 4fe5f15b8a6a4ebb9ce36f546d22cced
I20260808 07:13:50.124266 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 5633
I20260808 07:13:50.124251 5704 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:50 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40622
I20260808 07:13:50.125450 5862 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:50 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:48384
I20260808 07:13:50.125685 6132 tablet_replica.cc:330] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: stopping tablet replica
I20260808 07:13:50.125801 6132 raft_consensus.cc:2229] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:50.125942 6132 raft_consensus.cc:2258] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:50.126590 6012 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:50 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:41596
I20260808 07:13:50.126814 6133 tablet_replica.cc:330] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: stopping tablet replica
I20260808 07:13:50.126922 6133 raft_consensus.cc:2229] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:50.127056 6133 raft_consensus.cc:2258] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:50.127774 6133 ts_tablet_manager.cc:1905] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:50.128358 6132 ts_tablet_manager.cc:1905] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:50.129840 6133 ts_tablet_manager.cc:1918] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:50.129923 6133 log.cc:1196] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/4fe5f15b8a6a4ebb9ce36f546d22cced
I20260808 07:13:50.130213 6133 ts_tablet_manager.cc:1939] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 1517e22aabb6497d81322ee50b34c789: Deleting consensus metadata
I20260808 07:13:50.130198 6132 ts_tablet_manager.cc:1918] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:50.130369 6132 log.cc:1196] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/4fe5f15b8a6a4ebb9ce36f546d22cced
I20260808 07:13:50.130651 5545 catalog_manager.cc:4841] TS 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803): tablet 4fe5f15b8a6a4ebb9ce36f546d22cced (table test-table [id=1056f4e99e904b90bcded0b96837f131]) successfully deleted
I20260808 07:13:50.130776 6132 ts_tablet_manager.cc:1939] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 6c018e1ea4374710a59a5d792605a101: Deleting consensus metadata
I20260808 07:13:50.131148 5548 catalog_manager.cc:4841] TS 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017): tablet 4fe5f15b8a6a4ebb9ce36f546d22cced (table test-table [id=1056f4e99e904b90bcded0b96837f131]) successfully deleted
W20260808 07:13:50.133154 5547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:37145 (error 108)
I20260808 07:13:50.133518 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 5784
I20260808 07:13:50.133644 6099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.133523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37145 (local address 127.0.0.1:40636)
0808 07:13:50.133573 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.133608 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:13:50.135031 5547 catalog_manager.cc:4584] TS 83c908b58dde47e1b89e9faeab6b3cf3 (127.22.2.129:37145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37145: connect: Connection refused (error 111)
I20260808 07:13:50.141016 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 5943
I20260808 07:13:50.148155 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 5527
I20260808 07:13:50.155844 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44035
--webserver_interface=127.22.2.190
--webserver_port=42659
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:50.237941 6135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:50.238198 6135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:50.238237 6135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:50.239383 6135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:50.239454 6135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:50.239478 6135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:50.239508 6135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:50.239539 6135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:50.241008 6135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:44035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=42659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:50.241285 6135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:50.241514 6135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:50.250645 6141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:50.250645 6142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:50.250880 6135 server_base.cc:1034] running on GCE node
W20260808 07:13:50.250931 6144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:50.251209 6135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:50.251549 6135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:50.252740 6135 hybrid_clock.cc:648] HybridClock initialized: now 1786173230252722 us; error 41 us; skew 500 ppm
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:50.255296 6135 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:13:50.255761 6135 webserver.cc:460] Webserver started at http://127.22.2.190:42659/ using document root <none> and password file <none>
I20260808 07:13:50.255968 6135 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:50.256019 6135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:50.257305 6135 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:50.257920 6151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.258116 6135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:50.258191 6135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "aa31b50a1df444d0bd0ed440b5fea976"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "f07a9fab1f0da7455d859bd2f096cc5c"
server_key_iv: "deec98180a62a2f8aecdd4ce16659307"
server_key_version: "encryptionkey@0"
I20260808 07:13:50.258565 6135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:50.271152 6135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:50.272938 6135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:50.273051 6135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:50.277100 6135 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:44035
I20260808 07:13:50.277159 6210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:44035 every 8 connection(s)
I20260808 07:13:50.277539 6135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:50.279229 6211 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:13:50.279968 6211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Bootstrap starting.
I20260808 07:13:50.280952 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 6135
I20260808 07:13:50.281231 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:37145
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:44035
--webserver_port=43967
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:50.289615 6211 log.cc:824] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:50.291484 6211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:50.291808 6211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Bootstrap complete.
I20260808 07:13:50.294121 6211 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:50.294371 6211 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa31b50a1df444d0bd0ed440b5fea976, State: Initialized, Role: FOLLOWER
I20260808 07:13:50.294574 6211 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:50.294670 6211 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:50.294728 6211 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:50.294776 6211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:13:50.295877 6211 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:50.295994 6211 leader_election.cc:302] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa31b50a1df444d0bd0ed440b5fea976; no voters:
I20260808 07:13:50.296190 6211 leader_election.cc:288] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:13:50.296227 6215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:13:50.296361 6215 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 2 LEADER]: Becoming Leader. State: Replica: aa31b50a1df444d0bd0ed440b5fea976, State: Running, Role: LEADER
I20260808 07:13:50.296448 6215 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:50.296475 6211 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:50.296772 6216 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aa31b50a1df444d0bd0ed440b5fea976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } } }
I20260808 07:13:50.296867 6216 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:50.297088 6221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:50.297971 6221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:50.298046 6217 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa31b50a1df444d0bd0ed440b5fea976. Latest consensus state: current_term: 2 leader_uuid: "aa31b50a1df444d0bd0ed440b5fea976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } } }
I20260808 07:13:50.298094 6221 catalog_manager.cc:1259] Loaded cluster ID: dbd9bafd16d24b67ae12d08ec6214c44
I20260808 07:13:50.298113 6217 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:50.298120 6221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:50.298440 6221 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 07:13:50.303455 6221 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 07:13:50.306160 6221 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:50.306355 6221 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Loaded TSK: 0
I20260808 07:13:50.306561 6221 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:13:50.368688 6213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:50.368891 6213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:50.368916 6213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:50.370100 6213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:50.370143 6213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:50.370170 6213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:13:50.371822 6213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:37145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=43967
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:50.372063 6213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:50.372337 6213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:50.377532 6240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:50.377581 6243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:50.377902 6213 server_base.cc:1034] running on GCE node
W20260808 07:13:50.377931 6241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:50.378170 6213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:50.378472 6213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:50.379621 6213 hybrid_clock.cc:648] HybridClock initialized: now 1786173230379608 us; error 27 us; skew 500 ppm
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:50.382397 6213 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:13:50.382853 6213 webserver.cc:460] Webserver started at http://127.22.2.129:43967/ using document root <none> and password file <none>
I20260808 07:13:50.383108 6213 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:50.383193 6213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:50.384677 6213 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:50.385416 6252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.385671 6213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:50.385807 6213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "83c908b58dde47e1b89e9faeab6b3cf3"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "e85809b0b9f808fb915fec374f080277"
server_key_iv: "a1ce1ea2ad0ee109eed7af3834328028"
server_key_version: "encryptionkey@0"
I20260808 07:13:50.386206 6213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:50.422684 6213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:50.425135 6213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:50.425314 6213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:50.425565 6213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:50.426014 6261 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:13:50.427475 6213 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:13:50.427543 6213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:13:50.427611 6213 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:13:50.428284 6213 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:13:50.428321 6213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:50.428413 6261 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Bootstrap starting.
I20260808 07:13:50.435315 6213 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:37145
I20260808 07:13:50.435432 6384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:37145 every 8 connection(s)
I20260808 07:13:50.435775 6213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:50.437815 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 6213
I20260808 07:13:50.438127 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:35017
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:44035
--webserver_port=46467
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:50.439715 6261 log.cc:824] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:50.442191 6261 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:50.442492 6261 tablet_bootstrap.cc:492] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Bootstrap complete.
I20260808 07:13:50.443763 6261 ts_tablet_manager.cc:1397] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.012s
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:50.445050 6261 raft_consensus.cc:348] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:50.445313 6261 raft_consensus.cc:729] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c908b58dde47e1b89e9faeab6b3cf3, State: Initialized, Role: FOLLOWER
I20260808 07:13:50.445446 6261 consensus_queue.cc:260] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } }
I20260808 07:13:50.445838 6261 ts_tablet_manager.cc:1428] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:13:50.449365 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.436639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:54913 (local address 127.22.2.190:44035)
0808 07:13:50.436911 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.436926 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.436979 (+ 53us) server_negotiation.cc:407] Connection header received
0808 07:13:50.437544 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.437551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.437616 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.437779 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:50.438654 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.440066 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.441548 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.442201 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.442345 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.445610 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:50.445614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:50.445614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:50.445628 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:50.447514 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.448083 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.448085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.448086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.448173 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.448478 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.448479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.448479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.448684 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:50.448749 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.449109 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.449211 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":110,"threads_started":1}
I20260808 07:13:50.449502 6389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.436558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:54913)
0808 07:13:50.436807 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.436853 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.437297 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.437839 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.437843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.438232 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:50.438514 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.438521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.440195 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.440196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.441446 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.441448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.442465 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.442466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.442629 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.443269 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:50.443280 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:50.445443 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:50.447654 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:50.447662 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:50.447664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:50.447997 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:50.448244 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:50.448245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:50.448245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:50.448403 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:50.448755 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:50.448763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:50.448935 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.449058 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.449248 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":117,"threads_started":1}
I20260808 07:13:50.450244 6385 heartbeater.cc:347] Connected to a master server at 127.22.2.190:44035
I20260808 07:13:50.450358 6385 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:50.450555 6385 heartbeater.cc:511] Master 127.22.2.190:44035 requested a full tablet report, sending...
I20260808 07:13:50.451236 6172 ts_manager.cc:194] Registered new tserver with Master: 83c908b58dde47e1b89e9faeab6b3cf3 (127.22.2.129:37145)
I20260808 07:13:50.452307 6172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:54913
I20260808 07:13:50.453249 6385 heartbeater.cc:503] Master 127.22.2.190:44035 was elected leader, sending a full tablet report...
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:13:50.460359 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.451883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37145 (local address 127.0.0.1:40652)
0808 07:13:50.452092 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.452122 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.452180 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.452494 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.452500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.452841 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:50.452886 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.452889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.454585 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.454586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.455132 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.455133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.455762 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.455763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.455818 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.456291 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:50.456295 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:50.457520 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:50.459313 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:50.459319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:50.459321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:50.459505 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:50.459745 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:50.459745 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:50.459746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:50.459801 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:50.460092 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:50.460098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:50.460136 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.460197 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.460296 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":90,"threads_started":1}
I20260808 07:13:50.460434 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.451970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40652 (local address 127.22.2.129:37145)
0808 07:13:50.452299 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.452303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.452328 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:13:50.452359 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.452361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.452376 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.452467 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:50.453438 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.454507 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.455209 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.455677 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.455740 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.457598 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:50.457606 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:50.457606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:50.457613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:50.459231 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.459603 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.459606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.459606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.459670 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.459864 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.459865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.459865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.460028 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:50.460105 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.460243 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.460342 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":76,"threads_started":1}
I20260808 07:13:50.460804 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe5f15b8a6a4ebb9ce36f546d22cced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:50 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40652
I20260808 07:13:50.461372 6396 tablet_replica.cc:330] stopping tablet replica
I20260808 07:13:50.461488 6396 raft_consensus.cc:2229] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:50.461575 6396 raft_consensus.cc:2258] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:50.461941 6396 ts_tablet_manager.cc:1905] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:50.464005 6396 ts_tablet_manager.cc:1918] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:50.464092 6396 log.cc:1196] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/4fe5f15b8a6a4ebb9ce36f546d22cced
I20260808 07:13:50.464326 6396 ts_tablet_manager.cc:1939] T 4fe5f15b8a6a4ebb9ce36f546d22cced P 83c908b58dde47e1b89e9faeab6b3cf3: Deleting consensus metadata
I20260808 07:13:50.464861 6155 catalog_manager.cc:4841] TS 83c908b58dde47e1b89e9faeab6b3cf3 (127.22.2.129:37145): tablet 4fe5f15b8a6a4ebb9ce36f546d22cced (table test-table [id=1056f4e99e904b90bcded0b96837f131]) successfully deleted
W20260808 07:13:50.526208 6391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:50.526412 6391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:50.526435 6391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:50.527482 6391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:50.527514 6391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:50.527539 6391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:13:50.528988 6391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:35017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=46467
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:50.529161 6391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:50.529330 6391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:50.534291 6404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:50.534269 6407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:50.534286 6403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:50.534547 6391 server_base.cc:1034] running on GCE node
I20260808 07:13:50.534781 6391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:50.535024 6391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:50.536182 6391 hybrid_clock.cc:648] HybridClock initialized: now 1786173230536177 us; error 35 us; skew 500 ppm
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:50.538537 6391 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:13:50.539016 6391 webserver.cc:460] Webserver started at http://127.22.2.130:46467/ using document root <none> and password file <none>
I20260808 07:13:50.539196 6391 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:50.539327 6391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:50.540673 6391 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:50.541214 6415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.541365 6391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:50.541450 6391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "6c018e1ea4374710a59a5d792605a101"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "7dca1e6b606ef8866d46be9e9c09a75e"
server_key_iv: "f9d90caeb2afd8454b10e4ccaa17ccd0"
server_key_version: "encryptionkey@0"
I20260808 07:13:50.541742 6391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:50.567269 6391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:50.569675 6391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:50.569793 6391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:50.570000 6391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:50.570334 6391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:50.570372 6391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.570396 6391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:50.570449 6391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.576512 6391 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35017
I20260808 07:13:50.576594 6548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35017 every 8 connection(s)
I20260808 07:13:50.576941 6391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:50.584856 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 6391
I20260808 07:13:50.585146 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:40803
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:44035
--webserver_port=44551
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:50.586277 6551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.577656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:39707)
0808 07:13:50.577846 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.577882 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.578410 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.578663 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.578668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.579253 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:50.579445 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.579450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.580181 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.580182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.581011 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.581013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.581421 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.581423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.581584 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.582045 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:50.582053 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:50.583382 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:50.584840 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:50.584847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:50.584849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:50.585113 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:50.585344 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:50.585346 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:50.585346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:50.585469 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:50.585781 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:50.585786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:50.585900 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.585984 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.586101 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260808 07:13:50.586275 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.577682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:39707 (local address 127.22.2.190:44035)
0808 07:13:50.577749 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.577751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.577964 (+ 213us) server_negotiation.cc:407] Connection header received
0808 07:13:50.578526 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.578527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.578544 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.578596 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:50.579548 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.580078 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.581104 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.581333 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.581412 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.583482 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:50.583484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:50.583484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:50.583492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:50.584756 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.585191 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.585192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.585193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.585267 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.585544 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.585545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.585545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.585705 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:50.585787 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.586030 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.586133 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:13:50.587121 6549 heartbeater.cc:347] Connected to a master server at 127.22.2.190:44035
I20260808 07:13:50.587204 6549 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:50.587399 6549 heartbeater.cc:511] Master 127.22.2.190:44035 requested a full tablet report, sending...
I20260808 07:13:50.587817 6172 ts_manager.cc:194] Registered new tserver with Master: 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
I20260808 07:13:50.588333 6172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:39707
W20260808 07:13:50.669723 6552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:50.669929 6552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:50.669952 6552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:50.671015 6552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:50.671049 6552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:50.671121 6552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:13:50.673117 6552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:40803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=44551
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:50.673369 6552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:50.673596 6552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:50.678645 6559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:50.678690 6561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:50.678640 6558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:50.678877 6552 server_base.cc:1034] running on GCE node
I20260808 07:13:50.679073 6552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:50.679328 6552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:50.680487 6552 hybrid_clock.cc:648] HybridClock initialized: now 1786173230680477 us; error 44 us; skew 500 ppm
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:50.682988 6552 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:13:50.683408 6552 webserver.cc:460] Webserver started at http://127.22.2.131:44551/ using document root <none> and password file <none>
I20260808 07:13:50.683687 6552 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:50.683771 6552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:50.685115 6552 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:50.685667 6568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.685884 6552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:13:50.685981 6552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "1517e22aabb6497d81322ee50b34c789"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "31fc213f1a4e6f420150e95b0a0632b0"
server_key_iv: "6775f47b3450ea7bb86f07f4a71a4b19"
server_key_version: "encryptionkey@0"
I20260808 07:13:50.686331 6552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:50.703332 6552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:50.705683 6552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:50.705855 6552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:50.706094 6552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:50.706400 6552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:50.706455 6552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.706497 6552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:50.706524 6552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.713214 6552 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:40803
I20260808 07:13:50.713302 6696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:40803 every 8 connection(s)
I20260808 07:13:50.713676 6552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:13:50.721732 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 6552
I20260808 07:13:50.723084 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.714430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:47599 (local address 127.22.2.190:44035)
0808 07:13:50.714521 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.714524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.714706 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:13:50.715179 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.715180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.715191 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.715253 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:50.716359 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.716862 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.717821 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.718004 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.718053 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.720170 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:50.720171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:50.720172 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:50.720178 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:50.721602 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.722022 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.722023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.722024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.722086 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.722420 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.722421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.722422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.722546 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:50.722617 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.722908 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.722993 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:13:50.723274 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.714392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:47599)
0808 07:13:50.714581 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.714617 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.715083 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.715308 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.715311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.715951 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:50.716140 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.716145 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.716946 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.716947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.717723 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.717725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.718076 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.718077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.718176 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.718659 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:50.718667 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:50.720083 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:50.721682 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:50.721688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:50.721690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:50.721945 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:50.722177 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:50.722178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:50.722179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:50.722328 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:50.722622 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:50.722630 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:50.722766 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.722871 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.723033 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260808 07:13:50.723773 6697 heartbeater.cc:347] Connected to a master server at 127.22.2.190:44035
I20260808 07:13:50.723873 6697 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:50.724071 6697 heartbeater.cc:511] Master 127.22.2.190:44035 requested a full tablet report, sending...
I20260808 07:13:50.724514 6172 ts_manager.cc:194] Registered new tserver with Master: 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803)
I20260808 07:13:50.724946 6172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:47599
I20260808 07:13:50.728430 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.722156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.22.2.190:44035)
0808 07:13:50.722300 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.722301 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.722323 (+ 22us) server_negotiation.cc:407] Connection header received
0808 07:13:50.722387 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.722388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.722399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.722448 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:50.723004 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.723945 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.724844 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.725136 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.725199 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.726350 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:50.726351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:50.726351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:50.726357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:50.727494 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.727855 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.727856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.727856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.727909 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:50.728033 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:50.728033 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:50.728033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:50.728117 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:50.728220 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.728263 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.728341 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260808 07:13:50.729179 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:50.731770 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.729492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.22.2.190:44035)
0808 07:13:50.729556 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.729558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.729627 (+ 69us) server_negotiation.cc:407] Connection header received
0808 07:13:50.729670 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.729670 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.729698 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.729749 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:50.729849 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.730336 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.731038 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.731217 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.731270 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.731416 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:50.731499 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:50.731616 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.731665 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.731685 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:13:50.732010 6172 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:13:50.733476 6172 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:13:50.735473 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf (DEFAULT_TABLE table=test-table [id=28c5894024ac4f6997a245a7cd45b1e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:50.735716 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:50.737001 6261 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Bootstrap starting.
I20260808 07:13:50.737331 6261 tablet_bootstrap.cc:654] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:50.738251 6261 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: No bootstrap required, opened a new log
I20260808 07:13:50.738296 6261 ts_tablet_manager.cc:1397] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:50.738461 6261 raft_consensus.cc:348] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.738538 6261 raft_consensus.cc:374] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:50.738596 6261 raft_consensus.cc:729] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c908b58dde47e1b89e9faeab6b3cf3, State: Initialized, Role: FOLLOWER
I20260808 07:13:50.738648 6261 consensus_queue.cc:260] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.738747 6261 ts_tablet_manager.cc:1428] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.739488 6704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.735213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35017 (local address 127.0.0.1:48386)
0808 07:13:50.735376 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.735386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.735428 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.735910 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.735912 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.735915 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:50.735966 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.735968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.737315 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.737316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.738199 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.738202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.739067 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.739068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.739286 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.739314 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.739369 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.739418 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":5,"thread_start_us":111,"threads_started":1}
I20260808 07:13:50.739507 6706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.735470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48386 (local address 127.22.2.130:35017)
0808 07:13:50.735761 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.735766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.735775 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:50.735813 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.735814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.735828 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.735911 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:50.736034 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.737115 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.738286 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.738993 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.739071 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.739128 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.739390 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.739433 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":113,"threads_started":1}
I20260808 07:13:50.739601 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.735194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40803 (local address 127.0.0.1:41598)
0808 07:13:50.735248 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.735259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.735294 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.735809 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.735810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.735812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:50.735859 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.735860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.737243 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.737244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.738409 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.738412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.739314 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.739315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.739392 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.739396 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.739442 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.739486 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 07:13:50.739610 6705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.735392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.22.2.131:40803)
0808 07:13:50.735592 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.735598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.735609 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:13:50.735649 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.735651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.735666 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.735936 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:50.736000 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.737098 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.738497 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.739208 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.739309 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.739367 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.739466 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.739507 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260808 07:13:50.740423 6622 tablet_service.cc:1467] Processing CreateTablet for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf (DEFAULT_TABLE table=test-table [id=28c5894024ac4f6997a245a7cd45b1e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:50.740571 6472 tablet_service.cc:1467] Processing CreateTablet for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf (DEFAULT_TABLE table=test-table [id=28c5894024ac4f6997a245a7cd45b1e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:50.740799 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:50.740872 6472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:50.744362 6393 raft_consensus.cc:482] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:50.744478 6393 raft_consensus.cc:504] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.744513 6709 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: Bootstrap starting.
I20260808 07:13:50.744808 6393 leader_election.cc:288] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803), 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
I20260808 07:13:50.745210 6709 tablet_bootstrap.cc:654] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:50.745287 6708 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: Bootstrap starting.
I20260808 07:13:50.745955 6709 log.cc:824] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:50.746007 6708 tablet_bootstrap.cc:654] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:50.746486 6708 log.cc:824] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:50.747360 6709 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: No bootstrap required, opened a new log
I20260808 07:13:50.747433 6709 ts_tablet_manager.cc:1397] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:13:50.747927 6708 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: No bootstrap required, opened a new log
I20260808 07:13:50.748039 6708 ts_tablet_manager.cc:1397] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:13:50.748862 6389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.744954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40803 (local address 127.22.2.129:57451)
0808 07:13:50.745042 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.745055 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.745101 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.745483 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.745484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.745486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:50.745586 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.745589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.746726 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.746727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.747838 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.747842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.748576 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.748577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.748657 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.748678 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.748733 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.748774 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 07:13:50.749096 6706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.745005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:47355 (local address 127.22.2.130:35017)
0808 07:13:50.745164 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.745166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.745483 (+ 317us) server_negotiation.cc:407] Connection header received
0808 07:13:50.745650 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.745651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.745662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.745691 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:50.746000 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.746656 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.748146 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.748673 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.748725 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.748745 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.749000 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.749036 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260808 07:13:50.749182 6705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.745312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:57451 (local address 127.22.2.131:40803)
0808 07:13:50.745352 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.745353 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.745362 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:50.745383 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.745384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.745394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.745851 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:50.745885 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.746631 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.747944 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.748461 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.748534 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.748543 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.749086 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.749122 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:13:50.749464 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.744954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35017 (local address 127.22.2.129:47355)
0808 07:13:50.745426 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.745436 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:50.745506 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:50.745773 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:50.745774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:50.745776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:50.745918 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.745920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.746840 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.746841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.748039 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:50.748042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.748821 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.748823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:50.748905 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.748910 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:13:50.749258 (+ 348us) client_negotiation.cc:240] Negotiation successful
0808 07:13:50.749304 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":206,"thread_start_us":104,"threads_started":1}
I20260808 07:13:50.749538 6709 raft_consensus.cc:348] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.749688 6709 raft_consensus.cc:374] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:50.749730 6709 raft_consensus.cc:729] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c018e1ea4374710a59a5d792605a101, State: Initialized, Role: FOLLOWER
I20260808 07:13:50.749787 6496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ff2a7fa770c48218f40a19ff4f9b9cf" candidate_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c018e1ea4374710a59a5d792605a101" is_pre_election: true
I20260808 07:13:50.749845 6709 consensus_queue.cc:260] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.749404 6646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ff2a7fa770c48218f40a19ff4f9b9cf" candidate_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1517e22aabb6497d81322ee50b34c789" is_pre_election: true
I20260808 07:13:50.749948 6496 raft_consensus.cc:2454] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83c908b58dde47e1b89e9faeab6b3cf3 in term 0.
I20260808 07:13:50.750186 6709 ts_tablet_manager.cc:1428] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:13:50.750173 6257 leader_election.cc:302] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c018e1ea4374710a59a5d792605a101, 83c908b58dde47e1b89e9faeab6b3cf3; no voters:
W20260808 07:13:50.750264 6253 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803): Illegal state: must be running to vote when last-logged opid is not known
I20260808 07:13:50.750288 6393 raft_consensus.cc:2781] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:50.750299 6549 heartbeater.cc:503] Master 127.22.2.190:44035 was elected leader, sending a full tablet report...
I20260808 07:13:50.750331 6393 raft_consensus.cc:482] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:50.750357 6393 raft_consensus.cc:3037] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:50.750952 6708 raft_consensus.cc:348] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.751092 6708 raft_consensus.cc:374] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:50.751142 6708 raft_consensus.cc:729] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1517e22aabb6497d81322ee50b34c789, State: Initialized, Role: FOLLOWER
I20260808 07:13:50.751226 6393 raft_consensus.cc:504] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.751269 6708 consensus_queue.cc:260] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.751361 6393 leader_election.cc:288] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 election: Requested vote from peers 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803), 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
I20260808 07:13:50.751524 6646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ff2a7fa770c48218f40a19ff4f9b9cf" candidate_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1517e22aabb6497d81322ee50b34c789"
I20260808 07:13:50.751541 6496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ff2a7fa770c48218f40a19ff4f9b9cf" candidate_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c018e1ea4374710a59a5d792605a101"
I20260808 07:13:50.751595 6708 ts_tablet_manager.cc:1428] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:13:50.751606 6697 heartbeater.cc:503] Master 127.22.2.190:44035 was elected leader, sending a full tablet report...
I20260808 07:13:50.751636 6496 raft_consensus.cc:3037] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:50.751641 6646 raft_consensus.cc:3037] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:50.752305 6496 raft_consensus.cc:2454] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c908b58dde47e1b89e9faeab6b3cf3 in term 1.
I20260808 07:13:50.752406 6646 raft_consensus.cc:2454] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c908b58dde47e1b89e9faeab6b3cf3 in term 1.
I20260808 07:13:50.752467 6257 leader_election.cc:302] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c018e1ea4374710a59a5d792605a101, 83c908b58dde47e1b89e9faeab6b3cf3; no voters:
I20260808 07:13:50.752547 6393 raft_consensus.cc:2781] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:50.752699 6393 raft_consensus.cc:686] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 LEADER]: Becoming Leader. State: Replica: 83c908b58dde47e1b89e9faeab6b3cf3, State: Running, Role: LEADER
I20260808 07:13:50.752774 6393 consensus_queue.cc:237] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:50.753432 6171 catalog_manager.cc:5495] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83c908b58dde47e1b89e9faeab6b3cf3 (127.22.2.129). New cstate: current_term: 1 leader_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } health_report { overall_health: HEALTHY } } }
I20260808 07:13:50.762806 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.760533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40664 (local address 127.22.2.129:37145)
0808 07:13:50.760597 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:50.760599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:50.760609 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:50.760647 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:50.760648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:50.760663 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:50.760714 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:50.760873 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.761428 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.762118 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:50.762309 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:50.762357 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:50.762517 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:50.762598 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:50.762650 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:50.762691 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 07:13:50.762708 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:13:50.764248 6496 raft_consensus.cc:1261] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Refusing update from remote peer 83c908b58dde47e1b89e9faeab6b3cf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:50.764285 6646 raft_consensus.cc:1261] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Refusing update from remote peer 83c908b58dde47e1b89e9faeab6b3cf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:50.764590 6393 consensus_queue.cc:1035] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:50.764720 6715 consensus_queue.cc:1035] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:50.766714 6717 mvcc.cc:205] Tried to move back new op lower bound from 7316165553207873536 to 7316165553163849728. Current Snapshot: MvccSnapshot[applied={T|T < 7316165553207873536}]
I20260808 07:13:50.767315 6718 mvcc.cc:205] Tried to move back new op lower bound from 7316165553207873536 to 7316165553163849728. Current Snapshot: MvccSnapshot[applied={T|T < 7316165553207873536}]
I20260808 07:13:50.768965 6172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:50.769057 6172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:50.769361 6720 mvcc.cc:205] Tried to move back new op lower bound from 7316165553207873536 to 7316165553163849728. Current Snapshot: MvccSnapshot[applied={T|T < 7316165553207873536}]
I20260808 07:13:50.769793 6172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Sending DeleteTablet for 3 replicas of tablet 2ff2a7fa770c48218f40a19ff4f9b9cf
I20260808 07:13:50.770097 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:50 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:40652
I20260808 07:13:50.770099 6472 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:50 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:48386
I20260808 07:13:50.770184 6396 tablet_replica.cc:330] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: stopping tablet replica
I20260808 07:13:50.770241 6396 raft_consensus.cc:2229] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:13:50.770309 6396 raft_consensus.cc:2258] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:50.770519 6727 tablet_replica.cc:330] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: stopping tablet replica
I20260808 07:13:50.770617 6727 raft_consensus.cc:2229] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:50.770714 6727 raft_consensus.cc:2258] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:50.770699 6622 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:50 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:41598
I20260808 07:13:50.770892 6728 tablet_replica.cc:330] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: stopping tablet replica
I20260808 07:13:50.770911 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 6213
I20260808 07:13:50.773447 6728 raft_consensus.cc:2229] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:50.773561 6728 raft_consensus.cc:2258] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:50.774204 6727 ts_tablet_manager.cc:1905] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:50.776073 6728 ts_tablet_manager.cc:1905] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:50.776162 6727 ts_tablet_manager.cc:1918] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:50.776242 6727 log.cc:1196] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/2ff2a7fa770c48218f40a19ff4f9b9cf
I20260808 07:13:50.776523 6727 ts_tablet_manager.cc:1939] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 6c018e1ea4374710a59a5d792605a101: Deleting consensus metadata
I20260808 07:13:50.776966 6156 catalog_manager.cc:4841] TS 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017): tablet 2ff2a7fa770c48218f40a19ff4f9b9cf (table test-table [id=28c5894024ac4f6997a245a7cd45b1e2]) successfully deleted
I20260808 07:13:50.777486 6728 ts_tablet_manager.cc:1918] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:50.777575 6728 log.cc:1196] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/2ff2a7fa770c48218f40a19ff4f9b9cf
I20260808 07:13:50.777832 6728 ts_tablet_manager.cc:1939] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 1517e22aabb6497d81322ee50b34c789: Deleting consensus metadata
I20260808 07:13:50.778378 6152 catalog_manager.cc:4841] TS 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803): tablet 2ff2a7fa770c48218f40a19ff4f9b9cf (table test-table [id=28c5894024ac4f6997a245a7cd45b1e2]) successfully deleted
W20260808 07:13:50.779593 6155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:37145 (error 108)
I20260808 07:13:50.780103 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:50.779969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37145 (local address 127.0.0.1:40672)
0808 07:13:50.780028 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:50.780066 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:13:50.780287 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 6391
W20260808 07:13:50.780236 6155 catalog_manager.cc:4584] TS 83c908b58dde47e1b89e9faeab6b3cf3 (127.22.2.129:37145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ff2a7fa770c48218f40a19ff4f9b9cf: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37145: connect: Connection refused (error 111)
I20260808 07:13:50.788013 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 6552
I20260808 07:13:50.794121 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 6135
I20260808 07:13:50.801002 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44035
--webserver_interface=127.22.2.190
--webserver_port=42659
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:50.883736 6730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:50.883944 6730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:50.883966 6730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:50.885069 6730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:50.885104 6730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:50.885118 6730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:50.885131 6730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:50.885143 6730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:50.886541 6730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:44035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=42659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:50.886725 6730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:50.886890 6730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:13:50.892458 6730 server_base.cc:1034] running on GCE node
W20260808 07:13:50.892416 6736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:50.892403 6735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:50.892428 6738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:50.892935 6730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:50.893226 6730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:50.894392 6730 hybrid_clock.cc:648] HybridClock initialized: now 1786173230894384 us; error 48 us; skew 500 ppm
Aug 08 07:13:50 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173230, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:50.896837 6730 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:13:50.897253 6730 webserver.cc:460] Webserver started at http://127.22.2.190:42659/ using document root <none> and password file <none>
I20260808 07:13:50.897472 6730 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:50.897539 6730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:50.898883 6730 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:50.899551 6745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:50.899770 6730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:50.899868 6730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "aa31b50a1df444d0bd0ed440b5fea976"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "f07a9fab1f0da7455d859bd2f096cc5c"
server_key_iv: "deec98180a62a2f8aecdd4ce16659307"
server_key_version: "encryptionkey@0"
I20260808 07:13:50.900215 6730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:50.915827 6730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:50.917656 6730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:50.917826 6730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:50.922122 6730 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:44035
I20260808 07:13:50.922160 6808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:44035 every 8 connection(s)
I20260808 07:13:50.922555 6730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:50.924147 6810 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:13:50.924829 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Bootstrap starting.
I20260808 07:13:50.926622 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 6730
I20260808 07:13:50.926976 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:37145
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:44035
--webserver_port=43967
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:50.934824 6810 log.cc:824] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:50.937144 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:50.937464 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976: Bootstrap complete.
I20260808 07:13:50.939939 6810 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:50.940182 6810 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa31b50a1df444d0bd0ed440b5fea976, State: Initialized, Role: FOLLOWER
I20260808 07:13:50.940395 6810 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:50.940492 6810 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:50.940546 6810 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:50.940596 6810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:13:50.941655 6810 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:50.941774 6810 leader_election.cc:302] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa31b50a1df444d0bd0ed440b5fea976; no voters:
I20260808 07:13:50.941977 6810 leader_election.cc:288] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:13:50.942014 6816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:13:50.942122 6816 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [term 3 LEADER]: Becoming Leader. State: Replica: aa31b50a1df444d0bd0ed440b5fea976, State: Running, Role: LEADER
I20260808 07:13:50.942209 6816 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } }
I20260808 07:13:50.942247 6810 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:50.942493 6818 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "aa31b50a1df444d0bd0ed440b5fea976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } } }
I20260808 07:13:50.942787 6818 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:50.942893 6817 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa31b50a1df444d0bd0ed440b5fea976. Latest consensus state: current_term: 3 leader_uuid: "aa31b50a1df444d0bd0ed440b5fea976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa31b50a1df444d0bd0ed440b5fea976" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44035 } } }
I20260808 07:13:50.943123 6826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:50.943142 6817 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa31b50a1df444d0bd0ed440b5fea976 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:50.944063 6826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:50.944182 6826 catalog_manager.cc:1259] Loaded cluster ID: dbd9bafd16d24b67ae12d08ec6214c44
I20260808 07:13:50.944223 6826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 07:13:50.944738 6826 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f40478019ad google::LogMessage::Fail() at ??:0
@ 0x7f404780488a google::LogMessage::SendToLog() at ??:0
@ 0x7f40478014de google::LogMessage::Flush() at ??:0
@ 0x7f4047802a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f404f7220f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f404f72b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f40496cf0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f40496c4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f404a5286db start_thread at ??:0
@ 0x7f40462ff61f clone at ??:0
W20260808 07:13:51.026747 6814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:51.026975 6814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:51.027004 6814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:51.028398 6814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:51.028452 6814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:51.028482 6814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:13:51.030064 6814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:37145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=43967
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:51.030342 6814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:51.030629 6814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:51.035989 6853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:51.036103 6814 server_base.cc:1034] running on GCE node
W20260808 07:13:51.036029 6851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:51.036304 6850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:51.036528 6814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:51.036844 6814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:51.038020 6814 hybrid_clock.cc:648] HybridClock initialized: now 1786173231037994 us; error 44 us; skew 500 ppm
Aug 08 07:13:51 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173231, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:51.040911 6814 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:13:51.041425 6814 webserver.cc:460] Webserver started at http://127.22.2.129:43967/ using document root <none> and password file <none>
I20260808 07:13:51.041656 6814 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:51.041726 6814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:51.043170 6814 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:51.043972 6861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:51.044145 6814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:51.044265 6814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "83c908b58dde47e1b89e9faeab6b3cf3"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "e85809b0b9f808fb915fec374f080277"
server_key_iv: "a1ce1ea2ad0ee109eed7af3834328028"
server_key_version: "encryptionkey@0"
I20260808 07:13:51.044618 6814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:51.068449 6814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:51.070953 6814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:51.071132 6814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:51.071424 6814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:51.071975 6869 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:13:51.074221 6814 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:13:51.074296 6814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:13:51.074343 6814 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:13:51.075027 6814 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:13:51.075086 6814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:51.075182 6869 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Bootstrap starting.
I20260808 07:13:51.082664 6814 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:37145
I20260808 07:13:51.082818 6993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:37145 every 8 connection(s)
I20260808 07:13:51.083105 6814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:51.083389 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 6814
I20260808 07:13:51.083729 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:35017
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:44035
--webserver_port=46467
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:51.086964 6869 log.cc:824] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:51.089718 6869 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:51.090027 6869 tablet_bootstrap.cc:492] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Bootstrap complete.
I20260808 07:13:51.091044 6869 ts_tablet_manager.cc:1397] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260808 07:13:51.092182 6869 raft_consensus.cc:348] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:51.092437 6869 raft_consensus.cc:729] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c908b58dde47e1b89e9faeab6b3cf3, State: Initialized, Role: FOLLOWER
I20260808 07:13:51.092613 6869 consensus_queue.cc:260] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:51.092947 6869 ts_tablet_manager.cc:1428] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260808 07:13:51.159832 6998 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:13:51.086955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:45219)
0808 07:13:51.087180 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.087226 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:51.087851 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:51.159594 (+ 71743us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":111,"threads_started":1}
W20260808 07:13:51.160081 6996 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:44035 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:44035: Client connection negotiation failed: client connection to 127.22.2.190:44035: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:13:51.160518 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:51.160376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:57025)
0808 07:13:51.160437 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.160491 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:13:51.161029 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:51.160946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:58651)
0808 07:13:51.160990 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.161014 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:13:51.161158 6996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:13:51.188985 6995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:51.189201 6995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:51.189241 6995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:51.190353 6995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:51.190416 6995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:51.190462 6995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:13:51.192022 6995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:35017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=46467
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:51.192301 6995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:51.192541 6995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:51.197686 7007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:51.197649 7006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:51.197654 7010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:51.198015 6995 server_base.cc:1034] running on GCE node
I20260808 07:13:51.198182 6995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:51.198387 6995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:51.199528 6995 hybrid_clock.cc:648] HybridClock initialized: now 1786173231199514 us; error 28 us; skew 500 ppm
Aug 08 07:13:51 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173231, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:51.202150 6995 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:13:51.202564 6995 webserver.cc:460] Webserver started at http://127.22.2.130:46467/ using document root <none> and password file <none>
I20260808 07:13:51.202803 6995 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:51.202848 6995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:51.204206 6995 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:51.204816 7018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:51.204990 6995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:51.205080 6995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "6c018e1ea4374710a59a5d792605a101"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "7dca1e6b606ef8866d46be9e9c09a75e"
server_key_iv: "f9d90caeb2afd8454b10e4ccaa17ccd0"
server_key_version: "encryptionkey@0"
I20260808 07:13:51.205389 6995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:51.268693 6995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:51.271127 6995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:51.271282 6995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:51.271606 6995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:51.271989 6995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:51.272027 6995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:51.272080 6995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:51.272109 6995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:51.278189 6995 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35017
I20260808 07:13:51.278249 7151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35017 every 8 connection(s)
I20260808 07:13:51.278596 6995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:13:51.279670 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:51.279233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:50103)
0808 07:13:51.279400 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.279479 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
W20260808 07:13:51.280074 7152 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:44035 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:44035: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
I20260808 07:13:51.280390 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:51.280299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:37347)
0808 07:13:51.280345 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.280373 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:13:51.280692 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:51.280622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:33013)
0808 07:13:51.280657 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.280679 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 07:13:51.280845 7152 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:13:51.281854 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 6995
I20260808 07:13:51.282064 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:40803
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:44035
--webserver_port=44551
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:51.368664 7155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:51.368891 7155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:51.368918 7155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:51.370059 7155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:51.370107 7155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:51.370137 7155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:13:51.371589 7155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:40803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=44551
--tserver_master_addrs=127.22.2.190:44035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:13:51.371814 7155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:51.372011 7155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:51.377399 7163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:51.377460 7161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:51.377429 7160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:51.377624 7155 server_base.cc:1034] running on GCE node
I20260808 07:13:51.377861 7155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:51.378108 7155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:51.379251 7155 hybrid_clock.cc:648] HybridClock initialized: now 1786173231379232 us; error 39 us; skew 500 ppm
Aug 08 07:13:51 dist-test-slave-6k22 krb5kdc[5511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173231, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:51.381836 7155 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:13:51.382344 7155 webserver.cc:460] Webserver started at http://127.22.2.131:44551/ using document root <none> and password file <none>
I20260808 07:13:51.382602 7155 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:51.382676 7155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:51.384119 7155 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:51.384727 7173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:51.384933 7155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:51.385047 7155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "1517e22aabb6497d81322ee50b34c789"
format_stamp: "Formatted at 2026-08-08 07:13:49 on dist-test-slave-6k22"
server_key: "31fc213f1a4e6f420150e95b0a0632b0"
server_key_iv: "6775f47b3450ea7bb86f07f4a71a4b19"
server_key_version: "encryptionkey@0"
I20260808 07:13:51.385448 7155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:51.405362 7155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:51.407378 7155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:51.407631 7155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:51.407913 7155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:51.408275 7155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:51.408335 7155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:51.408370 7155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:51.408399 7155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:51.414631 7155 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:40803
I20260808 07:13:51.414700 7309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:40803 every 8 connection(s)
I20260808 07:13:51.415045 7155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:13:51.416188 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:51.415752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:52707)
0808 07:13:51.415933 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.416016 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
W20260808 07:13:51.416601 7310 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:44035 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:44035: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
I20260808 07:13:51.416956 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:51.416801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:42739)
0808 07:13:51.416900 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.416932 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 07:13:51.417253 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:51.417182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:51165)
0808 07:13:51.417205 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:51.417225 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260808 07:13:51.417368 7310 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:13:51.417877 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 7155
I20260808 07:13:52.161919 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:52.161624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:49449)
0808 07:13:52.161819 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:52.161863 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260808 07:13:52.281342 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:52.281134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:44155)
0808 07:13:52.281255 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:52.281289 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260808 07:13:52.417874 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:52.417660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:52175)
0808 07:13:52.417792 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:52.417825 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260808 07:13:52.587548 7317 raft_consensus.cc:482] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:52.587775 7317 raft_consensus.cc:504] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:52.588198 7317 leader_election.cc:288] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803), 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
Aug 08 07:13:52 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173231, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.130@KRBTEST.COM
Aug 08 07:13:52 dist-test-slave-6k22 krb5kdc[5511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173231, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.131@KRBTEST.COM
I20260808 07:13:52.599083 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:52.588375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35017 (local address 127.22.2.129:55503)
0808 07:13:52.588446 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:52.588458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:52.588515 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:52.589464 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:52.589467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:52.589878 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:52.590077 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:52.590082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:52.591176 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:52.591177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:52.592018 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:52.592021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:52.592637 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:52.592638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:52.592743 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:52.593363 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:52.593384 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:52.594971 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:52.597282 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:52.597288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:52.597290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:52.597542 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:52.597819 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:52.597820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:52.597820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:52.597935 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:52.598442 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:52.598451 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:52.598605 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 07:13:52.598747 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 07:13:52.598947 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":72}
I20260808 07:13:52.599105 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:52.588509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:55503 (local address 127.22.2.130:35017)
0808 07:13:52.588659 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:52.588677 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:52.588709 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:13:52.589212 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:52.589215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:52.589278 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:52.589480 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:52.590212 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:52.591066 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:52.592146 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:52.592542 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:52.592660 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:52.595081 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:52.595086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:52.595086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:52.595098 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:52.597195 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:52.597631 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:52.597634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:52.597634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:52.597718 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:52.598005 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:52.598006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:52.598006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:52.598325 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:52.598436 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:52.598855 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 07:13:52.598991 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260808 07:13:52.599105 7321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:52.588648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:60495 (local address 127.22.2.131:40803)
0808 07:13:52.588792 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:52.588800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:52.588836 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:13:52.589315 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:52.589317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:52.589355 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:52.589498 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:52.590210 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:52.591062 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:52.592113 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:52.592505 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:52.592654 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:52.595600 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:52.595604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:52.595604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:52.595615 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:52.597457 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:52.597783 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:52.597786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:52.597786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:52.597888 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:52.598089 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:52.598090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:52.598090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:52.598388 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:52.598463 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:52.598855 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 07:13:52.598991 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260808 07:13:52.599131 7318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:52.588400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:40803 (local address 127.22.2.129:60495)
0808 07:13:52.588528 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:52.588536 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:52.588576 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:52.589483 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:52.589484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:52.589878 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:52.590077 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:52.590082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:52.591168 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:52.591169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:52.592016 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:52.592019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:52.592631 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:52.592633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:52.592743 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:52.593427 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:52.593435 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:52.595459 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:52.597530 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:52.597531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:52.597531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:52.597710 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:52.597954 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:52.597954 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:52.597955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:52.598016 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:52.598462 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:52.598462 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:52.598605 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 07:13:52.598747 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 07:13:52.599004 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":141,"thread_start_us":82,"threads_started":1}
W20260808 07:13:52.600513 6862 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803): Not found: Tablet not found: 2ff2a7fa770c48218f40a19ff4f9b9cf
W20260808 07:13:52.600724 6865 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017): Not found: Tablet not found: 2ff2a7fa770c48218f40a19ff4f9b9cf
I20260808 07:13:52.600787 6865 leader_election.cc:302] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83c908b58dde47e1b89e9faeab6b3cf3; no voters: 1517e22aabb6497d81322ee50b34c789, 6c018e1ea4374710a59a5d792605a101
I20260808 07:13:52.600950 7317 raft_consensus.cc:2726] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:13:53.162659 7322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:53.162423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:44751)
0808 07:13:53.162565 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:53.162603 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260808 07:13:53.282025 7323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:53.281821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:60573)
0808 07:13:53.281944 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:53.281983 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260808 07:13:53.418627 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:53.418401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:56627)
0808 07:13:53.418544 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:53.418585 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260808 07:13:54.163442 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:54.163194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:42285)
0808 07:13:54.163345 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:54.163392 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260808 07:13:54.282780 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:54.282533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:39391)
0808 07:13:54.282688 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:54.282733 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260808 07:13:54.419406 7329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:54.419184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:56191)
0808 07:13:54.419327 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:54.419364 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260808 07:13:54.803821 7330 raft_consensus.cc:482] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:54.803927 7330 raft_consensus.cc:504] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:54.804184 7330 leader_election.cc:288] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803), 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
W20260808 07:13:54.804731 6862 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803): Not found: Tablet not found: 2ff2a7fa770c48218f40a19ff4f9b9cf
W20260808 07:13:54.804817 6865 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017): Not found: Tablet not found: 2ff2a7fa770c48218f40a19ff4f9b9cf
I20260808 07:13:54.804919 6865 leader_election.cc:302] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83c908b58dde47e1b89e9faeab6b3cf3; no voters: 1517e22aabb6497d81322ee50b34c789, 6c018e1ea4374710a59a5d792605a101
I20260808 07:13:54.805004 7330 raft_consensus.cc:2726] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:13:55.164170 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:55.163931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:37075)
0808 07:13:55.164082 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:55.164119 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260808 07:13:55.283514 7332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:55.283314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:56527)
0808 07:13:55.283434 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:55.283473 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260808 07:13:55.420202 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:55.420008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:56809)
0808 07:13:55.420126 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:55.420160 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260808 07:13:56.164921 7334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:56.164706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:50525)
0808 07:13:56.164837 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:56.164873 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260808 07:13:56.284205 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:56.284003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:39637)
0808 07:13:56.284122 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:56.284162 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260808 07:13:56.309255 7336 raft_consensus.cc:482] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:56.309371 7336 raft_consensus.cc:504] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:13:56.309595 7336 leader_election.cc:288] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803), 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
W20260808 07:13:56.310048 6862 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803): Not found: Tablet not found: 2ff2a7fa770c48218f40a19ff4f9b9cf
W20260808 07:13:56.310132 6865 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017): Not found: Tablet not found: 2ff2a7fa770c48218f40a19ff4f9b9cf
I20260808 07:13:56.310179 6865 leader_election.cc:302] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83c908b58dde47e1b89e9faeab6b3cf3; no voters: 1517e22aabb6497d81322ee50b34c789, 6c018e1ea4374710a59a5d792605a101
I20260808 07:13:56.310284 7336 raft_consensus.cc:2726] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:13:56.420918 7337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:56.420708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:36911)
0808 07:13:56.420836 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:56.420877 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260808 07:13:57.165683 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:57.165409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:40927)
0808 07:13:57.165569 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:57.165620 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260808 07:13:57.284965 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:57.284726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:47407)
0808 07:13:57.284877 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:57.284921 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260808 07:13:57.421718 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:57.421489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:45159)
0808 07:13:57.421616 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:57.421653 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260808 07:13:58.166445 7341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.166226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:37745)
0808 07:13:58.166359 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.166396 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260808 07:13:58.285885 7342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.285635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:44165)
0808 07:13:58.285798 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.285839 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":70,"threads_started":1}
I20260808 07:13:58.422435 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.422218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:54117)
0808 07:13:58.422354 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.422392 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260808 07:13:59.167197 7344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:59.166969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:49585)
0808 07:13:59.167114 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:59.167149 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260808 07:13:59.286775 7345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:59.286509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:52671)
0808 07:13:59.286661 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:59.286697 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260808 07:13:59.423161 7346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:59.422916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:48419)
0808 07:13:59.423079 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:59.423116 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
I20260808 07:14:00.167966 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:00.167719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:51139)
0808 07:14:00.167877 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:00.167916 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260808 07:14:00.287508 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:00.287292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:54775)
0808 07:14:00.287421 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:00.287461 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260808 07:14:00.423960 7349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:00.423751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:43307)
0808 07:14:00.423879 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:00.423917 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260808 07:14:01.168752 7352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:01.168490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:56597)
0808 07:14:01.168642 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:01.168681 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260808 07:14:01.202310 7353 raft_consensus.cc:482] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:01.202464 7353 raft_consensus.cc:504] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1517e22aabb6497d81322ee50b34c789" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 40803 } } peers { permanent_uuid: "6c018e1ea4374710a59a5d792605a101" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35017 } } peers { permanent_uuid: "83c908b58dde47e1b89e9faeab6b3cf3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37145 } }
I20260808 07:14:01.202795 7353 leader_election.cc:288] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803), 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017)
W20260808 07:14:01.203258 6865 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6c018e1ea4374710a59a5d792605a101 (127.22.2.130:35017): Not found: Tablet not found: 2ff2a7fa770c48218f40a19ff4f9b9cf
W20260808 07:14:01.203356 6862 leader_election.cc:341] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1517e22aabb6497d81322ee50b34c789 (127.22.2.131:40803): Not found: Tablet not found: 2ff2a7fa770c48218f40a19ff4f9b9cf
I20260808 07:14:01.203423 6862 leader_election.cc:302] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83c908b58dde47e1b89e9faeab6b3cf3; no voters: 1517e22aabb6497d81322ee50b34c789, 6c018e1ea4374710a59a5d792605a101
I20260808 07:14:01.203517 7353 raft_consensus.cc:2726] T 2ff2a7fa770c48218f40a19ff4f9b9cf P 83c908b58dde47e1b89e9faeab6b3cf3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:01.288479 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:01.288196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:51567)
0808 07:14:01.288370 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:01.288409 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
I20260808 07:14:01.424705 7355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:01.424444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:38833)
0808 07:14:01.424605 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:01.424651 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260808 07:14:02.169507 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:02.169267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:41693)
0808 07:14:02.169407 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:02.169456 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260808 07:14:02.289218 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:02.289009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:40317)
0808 07:14:02.289131 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:02.289166 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260808 07:14:02.425489 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:02.425269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:57283)
0808 07:14:02.425411 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:02.425445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260808 07:14:03.170315 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:03.170075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:54599)
0808 07:14:03.170229 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:03.170266 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260808 07:14:03.289954 7361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:03.289661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:36287)
0808 07:14:03.289860 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:03.289904 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
I20260808 07:14:03.426347 7362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:03.426072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:40093)
0808 07:14:03.426249 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:03.426294 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260808 07:14:04.170989 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:04.170763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:53787)
0808 07:14:04.170900 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:04.170939 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260808 07:14:04.290820 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:04.290563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:36113)
0808 07:14:04.290731 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:04.290771 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260808 07:14:04.427175 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:04.426908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:37547)
0808 07:14:04.427067 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:04.427116 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":63,"threads_started":1}
I20260808 07:14:05.171967 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:05.171675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:54099)
0808 07:14:05.171863 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:05.171916 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
I20260808 07:14:05.291661 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:05.291370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:45333)
0808 07:14:05.291503 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:05.291550 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260808 07:14:05.428040 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:05.427764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:43229)
0808 07:14:05.427939 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:05.427988 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":80,"threads_started":1}
I20260808 07:14:06.173174 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:06.172876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.129:40661)
0808 07:14:06.173039 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:06.173092 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260808 07:14:06.292507 7370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:06.292243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.130:45755)
0808 07:14:06.292411 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:06.292459 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":85,"threads_started":1}
I20260808 07:14:06.428951 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:06.428680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44035 (local address 127.22.2.131:54795)
0808 07:14:06.428862 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:06.428906 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
I20260808 07:14:06.429369 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 6814
I20260808 07:14:06.437786 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 6995
I20260808 07:14:06.444389 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 7155
2026-08-08T07:14:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18898 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:06 dist-test-slave-6k22 krb5kdc[7375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:06 dist-test-slave-6k22 krb5kdc[7375](info): set up 2 sockets
Aug 08 07:14:06 dist-test-slave-6k22 krb5kdc[7375](info): commencing operation
krb5kdc: starting...
W20260808 07:14:07.870435 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:14:07 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:14:07.996366 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35535
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:08.078683 7389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:08.078886 7389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:08.078908 7389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:08.080057 7389 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:14:08.080096 7389 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:14:08.080109 7389 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:14:08.080121 7389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:08.080134 7389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:08.080154 7389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:08.080169 7389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:08.080185 7389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:08.081780 7389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:08.081975 7389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:08.082149 7389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:14:08.087522 7389 server_base.cc:1034] running on GCE node
W20260808 07:14:08.087617 7397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:08.087484 7395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:08.087486 7399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:08.087913 7389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:08.088166 7389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:08.089309 7389 hybrid_clock.cc:648] HybridClock initialized: now 1786173248089280 us; error 43 us; skew 500 ppm
Aug 08 07:14:08 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173248, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:08.091904 7389 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:14:08.092367 7389 webserver.cc:460] Webserver started at http://127.22.2.190:42907/ using document root <none> and password file <none>
I20260808 07:14:08.092586 7389 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:08.092651 7389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:08.092800 7389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:08.093861 7389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "d86e64699ea74c44b1bfcc01e1c6800c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "c0d01cea5fc9d65ee6edf5f6a756d567"
server_key_iv: "2ec53e95bda7b7c494c0dff85ccbc035"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.094225 7389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "d86e64699ea74c44b1bfcc01e1c6800c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "c0d01cea5fc9d65ee6edf5f6a756d567"
server_key_iv: "2ec53e95bda7b7c494c0dff85ccbc035"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.095650 7389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:08.096376 7408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.096542 7389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:08.096638 7389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "d86e64699ea74c44b1bfcc01e1c6800c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "c0d01cea5fc9d65ee6edf5f6a756d567"
server_key_iv: "2ec53e95bda7b7c494c0dff85ccbc035"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.096719 7389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:08.107658 7389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:08.109405 7389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:08.109553 7389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:08.113839 7389 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35535
I20260808 07:14:08.113881 7465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35535 every 8 connection(s)
I20260808 07:14:08.114238 7389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:08.115190 7467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:08.117739 7467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Bootstrap starting.
I20260808 07:14:08.118362 7467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:08.118669 7467 log.cc:824] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:08.119354 7467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: No bootstrap required, opened a new log
I20260808 07:14:08.121129 7467 raft_consensus.cc:348] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:08.121286 7467 raft_consensus.cc:374] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:08.121311 7467 raft_consensus.cc:729] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86e64699ea74c44b1bfcc01e1c6800c, State: Initialized, Role: FOLLOWER
I20260808 07:14:08.121454 7467 consensus_queue.cc:260] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:08.121567 7467 raft_consensus.cc:388] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:08.121593 7467 raft_consensus.cc:482] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:08.121665 7467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:08.121724 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 7389
I20260808 07:14:08.121843 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:14:08.122031 22538 external_mini_cluster.cc:1428] Setting key eafa36c075e3fc74ccc7dfdc8d7cff4d
I20260808 07:14:08.122421 7467 raft_consensus.cc:504] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:08.122555 7467 leader_election.cc:302] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d86e64699ea74c44b1bfcc01e1c6800c; no voters:
I20260808 07:14:08.122737 7467 leader_election.cc:288] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:08.122958 7472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:08.123082 7467 sys_catalog.cc:564] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:08.123136 7472 raft_consensus.cc:686] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 1 LEADER]: Becoming Leader. State: Replica: d86e64699ea74c44b1bfcc01e1c6800c, State: Running, Role: LEADER
I20260808 07:14:08.123286 7472 consensus_queue.cc:237] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:08.123667 7473 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } } }
I20260808 07:14:08.123767 7474 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86e64699ea74c44b1bfcc01e1c6800c. Latest consensus state: current_term: 1 leader_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } } }
I20260808 07:14:08.123888 7474 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:08.123843 7473 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:08.124373 7476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:08.124888 7476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:08.126556 7476 catalog_manager.cc:1347] Generated new cluster ID: 4d158b476de84bf583674b4ff8a312a7
I20260808 07:14:08.126609 7476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:14:08 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:14:08.134138 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.122540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.22.2.190:35535)
0808 07:14:08.122748 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.122759 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.122792 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:14:08.122926 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.122934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.122995 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.123122 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:08.123844 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.125279 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.126370 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.126985 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.127083 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.130156 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:08.130161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:08.130161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:08.130175 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:08.132351 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.132729 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.132732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.132733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.132869 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.133070 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.133071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.133072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.133454 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:08.133677 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.133768 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.133919 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260808 07:14:08.137179 7476 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:08.137645 7476 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:08.141121 7476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Generated new TSK 0
I20260808 07:14:08.141283 7476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:14:08.170120 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35535
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:08.256273 7496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:08.256495 7496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:08.256533 7496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:08.257663 7496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:08.257725 7496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:08.257768 7496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:14:08.259236 7496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:08.259497 7496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:08.259744 7496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:08.265861 7502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:08.265923 7504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:08.265950 7501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:08.266290 7496 server_base.cc:1034] running on GCE node
I20260808 07:14:08.266505 7496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:08.266788 7496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:08.267957 7496 hybrid_clock.cc:648] HybridClock initialized: now 1786173248267936 us; error 35 us; skew 500 ppm
Aug 08 07:14:08 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173248, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:08.271095 7496 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:14:08.271597 7496 webserver.cc:460] Webserver started at http://127.22.2.129:42675/ using document root <none> and password file <none>
I20260808 07:14:08.271857 7496 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:08.271903 7496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:08.272058 7496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:08.273097 7496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "2f5393c2f0cf44648d76427d1c65f20c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "da5349e2ac2a26b1f7cd06703cc0d5c7"
server_key_iv: "c115102b20712a9b9322321ab9dd6e96"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.273445 7496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "2f5393c2f0cf44648d76427d1c65f20c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "da5349e2ac2a26b1f7cd06703cc0d5c7"
server_key_iv: "c115102b20712a9b9322321ab9dd6e96"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.274837 7496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:08.275660 7511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.275822 7496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:08.275919 7496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "2f5393c2f0cf44648d76427d1c65f20c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "da5349e2ac2a26b1f7cd06703cc0d5c7"
server_key_iv: "c115102b20712a9b9322321ab9dd6e96"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.275995 7496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:08.291818 7496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:08.294425 7496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:08.294597 7496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:08.294857 7496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:08.295202 7496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:08.295258 7496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.295315 7496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:08.295346 7496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.301923 7496 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:46653
I20260808 07:14:08.301967 7639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:46653 every 8 connection(s)
I20260808 07:14:08.302292 7496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:08.306043 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 7496
I20260808 07:14:08.306139 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:14:08.306347 22538 external_mini_cluster.cc:1428] Setting key f07963c886000c9bdde72c5a16eaffed
Aug 08 07:14:08 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173248, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:14:08.312238 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.302954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56149 (local address 127.22.2.190:35535)
0808 07:14:08.303023 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.303026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.303252 (+ 226us) server_negotiation.cc:407] Connection header received
0808 07:14:08.303819 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.303820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.303835 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.303895 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:08.304838 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.305334 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.306409 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.306726 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.306800 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.309192 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:08.309193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:08.309194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:08.309202 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:08.310647 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.311234 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.311235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.311235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.311285 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.311616 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.311616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.311616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.311718 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:08.311770 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.312107 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.312174 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:14:08.312554 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.302946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:56149)
0808 07:14:08.303127 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.303165 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:08.303685 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:08.303971 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:08.303974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:08.304556 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:08.304753 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.304758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.305408 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.305408 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.306307 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.306310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.306813 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.306814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.306978 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.307548 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:08.307596 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:08.309096 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:08.310733 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:08.310741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:08.310743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:08.311100 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:08.311353 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:08.311354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:08.311355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:08.311514 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:08.311783 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:08.311791 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:08.311963 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 07:14:08.312081 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 07:14:08.312269 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260808 07:14:08.313124 7640 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35535
I20260808 07:14:08.313234 7640 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:08.313453 7640 heartbeater.cc:511] Master 127.22.2.190:35535 requested a full tablet report, sending...
I20260808 07:14:08.313952 7427 ts_manager.cc:194] Registered new tserver with Master: 2f5393c2f0cf44648d76427d1c65f20c (127.22.2.129:46653)
I20260808 07:14:08.314533 7427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:56149
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:14:08.335633 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35535
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:08.425979 7645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:08.426267 7645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:08.426296 7645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:08.427596 7645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:08.427641 7645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:08.427702 7645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:14:08.429450 7645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:08.429728 7645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:08.429983 7645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:08.435171 7654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:08.435200 7652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:08.435380 7645 server_base.cc:1034] running on GCE node
W20260808 07:14:08.435263 7650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:08.435762 7645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:08.436026 7645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:08.437196 7645 hybrid_clock.cc:648] HybridClock initialized: now 1786173248437180 us; error 38 us; skew 500 ppm
Aug 08 07:14:08 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173248, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:08.439764 7645 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:14:08.440212 7645 webserver.cc:460] Webserver started at http://127.22.2.130:34001/ using document root <none> and password file <none>
I20260808 07:14:08.440418 7645 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:08.440469 7645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:08.440569 7645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:08.441619 7645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "e6748dc79f1448af871d9d035a40d24f"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "603ab1a26b87b6c81c3f072d3116458a"
server_key_iv: "404ce9518deedd96e12428565db4939c"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.441969 7645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "e6748dc79f1448af871d9d035a40d24f"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "603ab1a26b87b6c81c3f072d3116458a"
server_key_iv: "404ce9518deedd96e12428565db4939c"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.443295 7645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:08.444118 7662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.444273 7645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.444356 7645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "e6748dc79f1448af871d9d035a40d24f"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "603ab1a26b87b6c81c3f072d3116458a"
server_key_iv: "404ce9518deedd96e12428565db4939c"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.444411 7645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:08.453660 7645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:08.455876 7645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:08.455988 7645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:08.456180 7645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:08.456476 7645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:08.456511 7645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.456532 7645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:08.456544 7645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.462322 7645 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35669
I20260808 07:14:08.462379 7788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35669 every 8 connection(s)
I20260808 07:14:08.462682 7645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:14:08 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173248, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:14:08.471246 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 7645
I20260808 07:14:08.471367 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:14:08.471720 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.463402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:46269 (local address 127.22.2.190:35535)
0808 07:14:08.463445 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.463450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.463721 (+ 271us) server_negotiation.cc:407] Connection header received
0808 07:14:08.464209 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.464210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.464221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.464280 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:08.465225 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.465715 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.466699 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.466885 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.466930 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.469083 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:08.469085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:08.469085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:08.469092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:08.470332 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.470686 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.470686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.470686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.470737 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.470953 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.470954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.470954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.471068 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:08.471119 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.471540 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.471652 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:14:08.471730 22538 external_mini_cluster.cc:1428] Setting key 4a109b8841ad9ce236152d071b3c6fa0
I20260808 07:14:08.471997 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.463364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:46269)
0808 07:14:08.463531 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.463596 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:08.464120 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:08.464353 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:08.464356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:08.464957 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:08.465144 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.465148 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.465812 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.465813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.466614 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.466616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.466955 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.466956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.467064 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.467514 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:08.467522 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:08.468996 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:08.470402 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:08.470408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:08.470410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:08.470623 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:08.470798 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:08.470798 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:08.470798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:08.470902 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:08.471164 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:08.471173 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:08.471372 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 07:14:08.471526 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 07:14:08.471749 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260808 07:14:08.472631 7790 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35535
I20260808 07:14:08.472746 7790 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:08.472923 7790 heartbeater.cc:511] Master 127.22.2.190:35535 requested a full tablet report, sending...
I20260808 07:14:08.473313 7427 ts_manager.cc:194] Registered new tserver with Master: e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
I20260808 07:14:08.473693 7427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:46269
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:14:08.498425 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35535
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:08.583070 7795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:08.583273 7795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:08.583297 7795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:08.584462 7795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:08.584509 7795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:08.584559 7795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:14:08.585995 7795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:08.586237 7795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:08.586467 7795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:08.593071 7800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:08.593098 7801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:08.593261 7803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:08.593262 7795 server_base.cc:1034] running on GCE node
I20260808 07:14:08.593508 7795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:08.593788 7795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:08.594951 7795 hybrid_clock.cc:648] HybridClock initialized: now 1786173248594934 us; error 35 us; skew 500 ppm
Aug 08 07:14:08 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173248, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:08.597586 7795 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:14:08.598073 7795 webserver.cc:460] Webserver started at http://127.22.2.131:39907/ using document root <none> and password file <none>
I20260808 07:14:08.598300 7795 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:08.598356 7795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:08.598500 7795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:08.599640 7795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "0844581a67c74792a4d7312c06dc2fa3"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "764f9cdf369da3aaa549d19404c7a7d1"
server_key_iv: "fd47ceb1d55305177c07300b2237e50c"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.600031 7795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "0844581a67c74792a4d7312c06dc2fa3"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "764f9cdf369da3aaa549d19404c7a7d1"
server_key_iv: "fd47ceb1d55305177c07300b2237e50c"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.601383 7795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:14:08.602133 7813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.602288 7795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:14:08.602373 7795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "0844581a67c74792a4d7312c06dc2fa3"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "764f9cdf369da3aaa549d19404c7a7d1"
server_key_iv: "fd47ceb1d55305177c07300b2237e50c"
server_key_version: "encryptionkey@0"
I20260808 07:14:08.602440 7795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:08.611135 7795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:08.613301 7795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:08.613449 7795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:08.613699 7795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:08.614069 7795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:08.614107 7795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.614130 7795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:08.614166 7795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:08.621275 7795 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:38609
I20260808 07:14:08.621323 7942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:38609 every 8 connection(s)
I20260808 07:14:08.621616 7795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:08.623394 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 7795
I20260808 07:14:08.623486 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:14:08.623718 22538 external_mini_cluster.cc:1428] Setting key 5c65b6f51cb789808f63fbbe2eed8dfb
Aug 08 07:14:08 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173248, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:14:08.631173 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.622288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:43489 (local address 127.22.2.190:35535)
0808 07:14:08.622366 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.622369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.622544 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:14:08.623098 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.623099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.623109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.623159 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:08.624561 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.625221 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.626265 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.626444 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.626496 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.628687 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:08.628689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:08.628689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:08.628695 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:08.630093 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.630444 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.630445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.630445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.630495 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.630696 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.630697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.630697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.630795 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:08.630845 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.631051 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.631109 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:14:08.631372 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.622249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:43489)
0808 07:14:08.622420 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.622457 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:08.623011 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:08.623221 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:08.623225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:08.624115 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:08.624467 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.624475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.625305 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.625306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.626174 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.626177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.626520 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.626520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.626677 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.627193 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:08.627201 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:08.628597 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:08.630164 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:08.630169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:08.630171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:08.630387 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:08.630556 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:08.630556 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:08.630556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:08.630647 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:08.630844 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:08.630848 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:08.630951 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 07:14:08.631027 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:14:08.631136 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260808 07:14:08.631871 7943 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35535
I20260808 07:14:08.631951 7943 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:08.632095 7943 heartbeater.cc:511] Master 127.22.2.190:35535 requested a full tablet report, sending...
I20260808 07:14:08.632407 7427 ts_manager.cc:194] Registered new tserver with Master: 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609)
I20260808 07:14:08.632755 7427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:43489
I20260808 07:14:08.635730 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:14:08.642334 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.636931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.22.2.190:35535)
0808 07:14:08.636997 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.636999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.637077 (+ 78us) server_negotiation.cc:407] Connection header received
0808 07:14:08.637120 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.637121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.637128 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.637178 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:08.637648 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.638087 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.638754 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.638921 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.638968 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.640021 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:08.640022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:08.640022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:08.640028 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:08.641414 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.641709 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.641710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.641710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.641767 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:08.641898 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:08.641898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:08.641899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:08.642023 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:08.642095 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.642159 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.642229 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:14:08.643851 7427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:14:08.645413 7427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:14:08.651763 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.647544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38609 (local address 127.0.0.1:48926)
0808 07:14:08.647723 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.647761 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:08.647842 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:08.648088 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:08.648091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:08.648093 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:08.648160 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.648161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.649425 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.649427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.650514 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.650517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.651397 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.651398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.651491 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.651505 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:14:08.651627 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 07:14:08.651686 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260808 07:14:08.651773 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.647655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48926 (local address 127.22.2.131:38609)
0808 07:14:08.647812 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.647817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.647830 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:14:08.647974 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.647977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.647993 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.648105 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:08.648219 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.649318 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.650611 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.651315 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.651398 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.651456 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.651646 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.651687 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260808 07:14:08.651899 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.647774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46653 (local address 127.0.0.1:54776)
0808 07:14:08.648041 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.648051 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:08.648093 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:08.648429 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:08.648430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:08.648432 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:08.648473 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.648475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.649479 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.649480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.650478 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.650481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.651635 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.651636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.651700 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.651705 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:14:08.651761 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:14:08.651820 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":19,"thread_start_us":85,"threads_started":1}
I20260808 07:14:08.652129 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.648022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.22.2.129:46653)
0808 07:14:08.648260 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.648265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.648274 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:08.648323 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.648325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.648339 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.648424 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:08.648566 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.649406 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.650571 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.651072 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.651162 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.651231 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.651970 (+ 739us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.652020 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":102,"threads_started":1}
I20260808 07:14:08.652292 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.647616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35669 (local address 127.0.0.1:50368)
0808 07:14:08.647899 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.647911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:08.647965 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:08.648721 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:08.648722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:08.648724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:08.648768 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.648770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.650011 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.650012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.651066 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.651069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.651910 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.651911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.652113 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.652117 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:08.652170 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:14:08.652214 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":88,"threads_started":1}
I20260808 07:14:08.652330 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.647859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.22.2.130:35669)
0808 07:14:08.648572 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.648577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.648586 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:08.648622 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.648623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.648637 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.648712 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:08.648848 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.649904 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.651152 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.651768 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.651826 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.651875 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.652197 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.652255 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":85,"threads_started":1}
I20260808 07:14:08.652660 7868 tablet_service.cc:1467] Processing CreateTablet for tablet 20e974de79684426b576f69db380646b (DEFAULT_TABLE table=test-table [id=e2f58cbe71c54e5cbd55a9dab9352aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:08.652801 7567 tablet_service.cc:1467] Processing CreateTablet for tablet 20e974de79684426b576f69db380646b (DEFAULT_TABLE table=test-table [id=e2f58cbe71c54e5cbd55a9dab9352aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:08.653056 7868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20e974de79684426b576f69db380646b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:08.653096 7567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20e974de79684426b576f69db380646b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:08.656152 7718 tablet_service.cc:1467] Processing CreateTablet for tablet 20e974de79684426b576f69db380646b (DEFAULT_TABLE table=test-table [id=e2f58cbe71c54e5cbd55a9dab9352aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:08.656495 7718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20e974de79684426b576f69db380646b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:08.656605 7964 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Bootstrap starting.
I20260808 07:14:08.657203 7965 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: Bootstrap starting.
I20260808 07:14:08.657588 7964 tablet_bootstrap.cc:654] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:08.657859 7965 tablet_bootstrap.cc:654] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:08.657903 7964 log.cc:824] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:08.658465 7965 log.cc:824] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:08.658635 7964 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: No bootstrap required, opened a new log
I20260808 07:14:08.658754 7964 ts_tablet_manager.cc:1397] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:14:08.659222 7965 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: No bootstrap required, opened a new log
I20260808 07:14:08.659296 7965 ts_tablet_manager.cc:1397] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:14:08.659852 7969 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Bootstrap starting.
I20260808 07:14:08.660427 7969 tablet_bootstrap.cc:654] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:08.660928 7969 log.cc:824] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:08.660948 7964 raft_consensus.cc:348] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.661221 7964 raft_consensus.cc:374] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:08.661252 7965 raft_consensus.cc:348] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.661288 7964 raft_consensus.cc:729] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5393c2f0cf44648d76427d1c65f20c, State: Initialized, Role: FOLLOWER
I20260808 07:14:08.661439 7965 raft_consensus.cc:374] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:08.661509 7965 raft_consensus.cc:729] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0844581a67c74792a4d7312c06dc2fa3, State: Initialized, Role: FOLLOWER
I20260808 07:14:08.661556 7964 consensus_queue.cc:260] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.661648 7969 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: No bootstrap required, opened a new log
I20260808 07:14:08.661628 7965 consensus_queue.cc:260] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.661742 7969 ts_tablet_manager.cc:1397] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:14:08.661850 7640 heartbeater.cc:503] Master 127.22.2.190:35535 was elected leader, sending a full tablet report...
I20260808 07:14:08.661859 7964 ts_tablet_manager.cc:1428] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:08.661971 7965 ts_tablet_manager.cc:1428] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:14:08.661986 7943 heartbeater.cc:503] Master 127.22.2.190:35535 was elected leader, sending a full tablet report...
I20260808 07:14:08.663889 7969 raft_consensus.cc:348] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.664033 7969 raft_consensus.cc:374] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:08.664078 7969 raft_consensus.cc:729] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6748dc79f1448af871d9d035a40d24f, State: Initialized, Role: FOLLOWER
I20260808 07:14:08.664187 7969 consensus_queue.cc:260] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.664458 7969 ts_tablet_manager.cc:1428] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:14:08.664469 7790 heartbeater.cc:503] Master 127.22.2.190:35535 was elected leader, sending a full tablet report...
W20260808 07:14:08.713261 7791 tablet.cc:2369] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:14:08.728245 7973 raft_consensus.cc:482] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:08.728379 7973 raft_consensus.cc:504] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.728698 7973 leader_election.cc:288] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669), 2f5393c2f0cf44648d76427d1c65f20c (127.22.2.129:46653)
I20260808 07:14:08.731642 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.728960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:58475 (local address 127.22.2.129:46653)
0808 07:14:08.729008 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.729010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.729202 (+ 192us) server_negotiation.cc:407] Connection header received
0808 07:14:08.729247 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.729248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.729261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.729321 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:08.729486 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.729960 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.730895 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.731268 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.731357 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.731366 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.731501 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.731542 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:14:08.731654 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.728882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35669 (local address 127.22.2.131:59381)
0808 07:14:08.728940 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.728952 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:08.728999 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:08.729182 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:08.729183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:08.729185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:08.729266 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.729269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.729980 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.729981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.730742 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.730744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.731339 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.731341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.731427 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.731430 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:08.731494 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:14:08.731542 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":5}
I20260808 07:14:08.731649 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.728963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:59381 (local address 127.22.2.130:35669)
0808 07:14:08.729008 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.729010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.729021 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:14:08.729057 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.729058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.729072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.729148 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:08.729351 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.729878 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.730830 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.731246 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.731302 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.731309 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.731500 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.731540 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:14:08.731675 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.728882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46653 (local address 127.22.2.131:58475)
0808 07:14:08.729121 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.729132 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:08.729192 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:08.729336 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:08.729337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:08.729338 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:08.729416 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.729418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.730030 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.730031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.730804 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:08.730807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.731371 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.731372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:08.731426 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.731430 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:08.731494 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:14:08.731539 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":13,"thread_start_us":144,"threads_started":1}
I20260808 07:14:08.732023 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20e974de79684426b576f69db380646b" candidate_uuid: "0844581a67c74792a4d7312c06dc2fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6748dc79f1448af871d9d035a40d24f" is_pre_election: true
I20260808 07:14:08.732069 7588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20e974de79684426b576f69db380646b" candidate_uuid: "0844581a67c74792a4d7312c06dc2fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f5393c2f0cf44648d76427d1c65f20c" is_pre_election: true
I20260808 07:14:08.732182 7741 raft_consensus.cc:2454] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0844581a67c74792a4d7312c06dc2fa3 in term 0.
I20260808 07:14:08.732268 7588 raft_consensus.cc:2454] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0844581a67c74792a4d7312c06dc2fa3 in term 0.
I20260808 07:14:08.732430 7817 leader_election.cc:302] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0844581a67c74792a4d7312c06dc2fa3, e6748dc79f1448af871d9d035a40d24f; no voters:
I20260808 07:14:08.732577 7973 raft_consensus.cc:2781] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:14:08.732647 7973 raft_consensus.cc:482] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:14:08.732682 7973 raft_consensus.cc:3037] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:08.733413 7973 raft_consensus.cc:504] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.733559 7973 leader_election.cc:288] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [CANDIDATE]: Term 1 election: Requested vote from peers e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669), 2f5393c2f0cf44648d76427d1c65f20c (127.22.2.129:46653)
I20260808 07:14:08.733731 7588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20e974de79684426b576f69db380646b" candidate_uuid: "0844581a67c74792a4d7312c06dc2fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f5393c2f0cf44648d76427d1c65f20c"
I20260808 07:14:08.733771 7741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20e974de79684426b576f69db380646b" candidate_uuid: "0844581a67c74792a4d7312c06dc2fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6748dc79f1448af871d9d035a40d24f"
I20260808 07:14:08.733827 7588 raft_consensus.cc:3037] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:08.733878 7741 raft_consensus.cc:3037] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:08.734542 7588 raft_consensus.cc:2454] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0844581a67c74792a4d7312c06dc2fa3 in term 1.
I20260808 07:14:08.734715 7815 leader_election.cc:302] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0844581a67c74792a4d7312c06dc2fa3, 2f5393c2f0cf44648d76427d1c65f20c; no voters:
I20260808 07:14:08.734786 7741 raft_consensus.cc:2454] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0844581a67c74792a4d7312c06dc2fa3 in term 1.
I20260808 07:14:08.734819 7973 raft_consensus.cc:2781] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:08.734998 7973 raft_consensus.cc:686] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 0844581a67c74792a4d7312c06dc2fa3, State: Running, Role: LEADER
I20260808 07:14:08.735100 7973 consensus_queue.cc:237] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:08.736014 7427 catalog_manager.cc:5495] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "0844581a67c74792a4d7312c06dc2fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } health_report { overall_health: UNKNOWN } } }
I20260808 07:14:08.742890 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.740548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48938 (local address 127.22.2.131:38609)
0808 07:14:08.740608 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:08.740610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:08.740619 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:08.740671 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:08.740672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:08.740689 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:08.740769 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:08.740907 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.741381 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.742131 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:08.742331 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:08.742401 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:08.742564 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:08.742657 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:14:08.742740 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:08.742778 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 07:14:08.742796 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:14:08.744607 7741 raft_consensus.cc:1261] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Refusing update from remote peer 0844581a67c74792a4d7312c06dc2fa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:08.744608 7588 raft_consensus.cc:1261] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Refusing update from remote peer 0844581a67c74792a4d7312c06dc2fa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:08.744949 7977 consensus_queue.cc:1035] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:08.745087 7973 consensus_queue.cc:1035] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:08.747545 7980 mvcc.cc:205] Tried to move back new op lower bound from 7316165626855485440 to 7316165626819698688. Current Snapshot: MvccSnapshot[applied={T|T < 7316165626855485440}]
I20260808 07:14:08.748015 7982 mvcc.cc:205] Tried to move back new op lower bound from 7316165626855485440 to 7316165626819698688. Current Snapshot: MvccSnapshot[applied={T|T < 7316165626855485440}]
I20260808 07:14:08.750844 7425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:08.750922 7425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:08.751684 7425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Sending DeleteTablet for 3 replicas of tablet 20e974de79684426b576f69db380646b
I20260808 07:14:08.752043 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 7496
I20260808 07:14:08.752049 7868 tablet_service.cc:1514] Processing DeleteTablet for tablet 20e974de79684426b576f69db380646b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:08 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:48926
I20260808 07:14:08.752033 7567 tablet_service.cc:1514] Processing DeleteTablet for tablet 20e974de79684426b576f69db380646b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:08 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:54776
I20260808 07:14:08.752259 7718 tablet_service.cc:1514] Processing DeleteTablet for tablet 20e974de79684426b576f69db380646b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:08 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:50368
I20260808 07:14:08.759704 7991 tablet_replica.cc:330] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: stopping tablet replica
I20260808 07:14:08.759841 7991 raft_consensus.cc:2229] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:08.759972 7991 raft_consensus.cc:2258] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:08.759974 7990 tablet_replica.cc:330] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: stopping tablet replica
I20260808 07:14:08.760159 7990 raft_consensus.cc:2229] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 1 LEADER]: Raft consensus shutting down.
W20260808 07:14:08.760203 7410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:46653 (error 108)
I20260808 07:14:08.760381 7990 raft_consensus.cc:2258] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:08.760654 7991 ts_tablet_manager.cc:1905] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:08.760885 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 7645
I20260808 07:14:08.764148 7990 ts_tablet_manager.cc:1905] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:08.766134 7990 ts_tablet_manager.cc:1918] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:08.766247 7990 log.cc:1196] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/20e974de79684426b576f69db380646b
I20260808 07:14:08.766556 7990 ts_tablet_manager.cc:1939] T 20e974de79684426b576f69db380646b P 0844581a67c74792a4d7312c06dc2fa3: Deleting consensus metadata
I20260808 07:14:08.768334 7410 catalog_manager.cc:4841] TS 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609): tablet 20e974de79684426b576f69db380646b (table test-table [id=e2f58cbe71c54e5cbd55a9dab9352aab]) successfully deleted
I20260808 07:14:08.768500 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.768284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46653 (local address 127.0.0.1:54792)
0808 07:14:08.768432 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.768468 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:46653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260808 07:14:08.769930 7410 catalog_manager.cc:4584] TS 2f5393c2f0cf44648d76427d1c65f20c (127.22.2.129:46653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20e974de79684426b576f69db380646b: Network error: Client connection negotiation failed: client connection to 127.22.2.129:46653: connect: Connection refused (error 111)
I20260808 07:14:08.776935 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:08.776833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35669 (local address 127.0.0.1:50374)
0808 07:14:08.776875 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:08.776901 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.130:35669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 07:14:08.776927 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 7795
I20260808 07:14:08.782943 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 7389
I20260808 07:14:28.789633 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35535
--webserver_interface=127.22.2.190
--webserver_port=42907
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:28.874119 7994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:28.874356 7994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:28.874392 7994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:28.875538 7994 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:14:28.875655 7994 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:14:28.875681 7994 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:14:28.875710 7994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:28.875738 7994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:28.875770 7994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:28.875797 7994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:28.875826 7994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:28.877362 7994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=42907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:28.877646 7994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:28.877892 7994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:28.887733 8002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:28.887740 8001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:28.887831 7994 server_base.cc:1034] running on GCE node
W20260808 07:14:28.887758 8005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:28.888307 7994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:28.888589 7994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:28.889730 7994 hybrid_clock.cc:648] HybridClock initialized: now 1786173268889715 us; error 29 us; skew 500 ppm
Aug 08 07:14:28 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173268, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:28.892351 7994 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:14:28.892808 7994 webserver.cc:460] Webserver started at http://127.22.2.190:42907/ using document root <none> and password file <none>
I20260808 07:14:28.893064 7994 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:28.893148 7994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:28.894506 7994 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:28.895184 8012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:28.895380 7994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:14:28.895491 7994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "d86e64699ea74c44b1bfcc01e1c6800c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "c0d01cea5fc9d65ee6edf5f6a756d567"
server_key_iv: "2ec53e95bda7b7c494c0dff85ccbc035"
server_key_version: "encryptionkey@0"
I20260808 07:14:28.895938 7994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:28.908583 7994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:28.910416 7994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:28.910575 7994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:28.914774 7994 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35535
I20260808 07:14:28.915036 8072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35535 every 8 connection(s)
I20260808 07:14:28.915259 7994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:28.915764 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 7994
I20260808 07:14:28.916069 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:46653
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:35535
--webserver_port=42675
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:14:28.916981 8073 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:14:28.917668 8073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Bootstrap starting.
I20260808 07:14:28.927474 8073 log.cc:824] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:28.929714 8073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:14:28.930109 8073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Bootstrap complete.
I20260808 07:14:28.932646 8073 raft_consensus.cc:348] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:28.932909 8073 raft_consensus.cc:729] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86e64699ea74c44b1bfcc01e1c6800c, State: Initialized, Role: FOLLOWER
I20260808 07:14:28.933106 8073 consensus_queue.cc:260] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:28.933189 8073 raft_consensus.cc:388] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:28.933249 8073 raft_consensus.cc:482] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:28.933313 8073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:14:28.934345 8073 raft_consensus.cc:504] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:28.934469 8073 leader_election.cc:302] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d86e64699ea74c44b1bfcc01e1c6800c; no voters:
I20260808 07:14:28.934677 8073 leader_election.cc:288] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:14:28.934715 8077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:14:28.934844 8077 raft_consensus.cc:686] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 2 LEADER]: Becoming Leader. State: Replica: d86e64699ea74c44b1bfcc01e1c6800c, State: Running, Role: LEADER
I20260808 07:14:28.934985 8073 sys_catalog.cc:564] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:28.935001 8077 consensus_queue.cc:237] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:28.935474 8078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } } }
I20260808 07:14:28.935803 8078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:28.935717 8079 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86e64699ea74c44b1bfcc01e1c6800c. Latest consensus state: current_term: 2 leader_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } } }
I20260808 07:14:28.936021 8079 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:28.936916 8096 catalog_manager.cc:1259] Loaded cluster ID: 4d158b476de84bf583674b4ff8a312a7
I20260808 07:14:28.936968 8096 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: loading cluster ID for follower catalog manager: success
I20260808 07:14:28.938186 8096 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: acquiring CA information for follower catalog manager: success
I20260808 07:14:28.938817 8096 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:14:28.938892 8092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:28.939323 8092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:28.939435 8092 catalog_manager.cc:1259] Loaded cluster ID: 4d158b476de84bf583674b4ff8a312a7
I20260808 07:14:28.939459 8092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:28.939666 8092 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:28.939751 8092 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Loaded TSK: 0
I20260808 07:14:28.964139 8092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Generated new TSK 1
I20260808 07:14:28.964262 8092 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:14:29.005071 8075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:29.005311 8075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:29.005352 8075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:29.006498 8075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:29.006564 8075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:29.006611 8075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:14:29.008194 8075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:46653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=42675
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:29.008462 8075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:29.008699 8075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:29.014808 8105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:29.014885 8107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:29.015007 8075 server_base.cc:1034] running on GCE node
W20260808 07:14:29.015034 8104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:29.015295 8075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:29.015623 8075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:29.016819 8075 hybrid_clock.cc:648] HybridClock initialized: now 1786173269016790 us; error 41 us; skew 500 ppm
Aug 08 07:14:29 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173269, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:29.019994 8075 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:14:29.020524 8075 webserver.cc:460] Webserver started at http://127.22.2.129:42675/ using document root <none> and password file <none>
I20260808 07:14:29.020800 8075 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:29.020861 8075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:29.022271 8075 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:14:29.023061 8116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:29.023245 8075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:29.023391 8075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "2f5393c2f0cf44648d76427d1c65f20c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "da5349e2ac2a26b1f7cd06703cc0d5c7"
server_key_iv: "c115102b20712a9b9322321ab9dd6e96"
server_key_version: "encryptionkey@0"
I20260808 07:14:29.023816 8075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:29.049978 8075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:29.052528 8075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:29.052767 8075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:29.053068 8075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:29.053596 8123 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:14:29.055090 8075 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:14:29.055143 8075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:14:29.055181 8075 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:14:29.055956 8075 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:14:29.056002 8075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:29.056044 8123 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Bootstrap starting.
I20260808 07:14:29.063983 8075 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:46653
I20260808 07:14:29.064059 8244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:46653 every 8 connection(s)
I20260808 07:14:29.064628 8075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:29.068218 8123 log.cc:824] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:29.071961 8123 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:14:29.072345 8123 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Bootstrap complete.
I20260808 07:14:29.073611 8123 ts_tablet_manager.cc:1397] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.012s
Aug 08 07:14:29 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173269, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:14:29.074076 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 8075
I20260808 07:14:29.074378 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:35669
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:35535
--webserver_port=34001
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:14:29.075103 8123 raft_consensus.cc:348] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.075495 8123 raft_consensus.cc:729] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5393c2f0cf44648d76427d1c65f20c, State: Initialized, Role: FOLLOWER
I20260808 07:14:29.075721 8123 consensus_queue.cc:260] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.076103 8123 ts_tablet_manager.cc:1428] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
I20260808 07:14:29.078711 8247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.065380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:32899)
0808 07:14:29.065674 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.065715 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:29.066302 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:29.067011 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:29.067016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:29.067484 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:29.067824 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.067831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.069589 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.069590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.070775 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.070786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.071596 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.071598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.071752 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.072383 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:29.072393 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:29.074323 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:29.076686 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:29.076693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:29.076696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:29.077029 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:29.077353 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:29.077354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:29.077354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:29.077527 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:29.078059 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:29.078066 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:29.078228 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 07:14:29.078340 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:14:29.078523 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":102,"threads_started":1}
I20260808 07:14:29.079013 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.065525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:32899 (local address 127.22.2.190:35535)
0808 07:14:29.066524 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.066533 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.066573 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:14:29.066716 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.066723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.066784 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.066935 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:29.067965 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.069448 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.070886 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.071458 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.071611 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.074453 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:29.074458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:29.074458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:29.074470 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:29.076567 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.077135 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.077139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.077139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.077259 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.077618 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.077619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.077619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.077968 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:29.078058 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.078430 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.078587 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":118,"threads_started":1}
I20260808 07:14:29.079663 8245 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35535
I20260808 07:14:29.079764 8245 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:29.079936 8245 heartbeater.cc:511] Master 127.22.2.190:35535 requested a full tablet report, sending...
I20260808 07:14:29.080739 8029 ts_manager.cc:194] Registered new tserver with Master: 2f5393c2f0cf44648d76427d1c65f20c (127.22.2.129:46653)
I20260808 07:14:29.082288 8029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:32899
I20260808 07:14:29.083290 8245 heartbeater.cc:503] Master 127.22.2.190:35535 was elected leader, sending a full tablet report...
Aug 08 07:14:29 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173268, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.129@KRBTEST.COM
I20260808 07:14:29.090399 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.081556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46653 (local address 127.0.0.1:45710)
0808 07:14:29.081847 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.081874 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:29.081971 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:29.082199 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:29.082205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:29.082556 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:29.082636 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.082639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.083820 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.083821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.084838 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.084841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.085321 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.085323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.085412 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.085895 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:29.085901 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:29.087443 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:29.089193 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:29.089201 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:29.089202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:29.089409 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:29.089637 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:29.089638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:29.089638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:29.089719 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:29.090012 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:29.090020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:29.090071 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 07:14:29.090173 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:14:29.090296 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":97,"threads_started":1}
I20260808 07:14:29.090400 8254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.081623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45710 (local address 127.22.2.129:46653)
0808 07:14:29.081791 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.081795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.081950 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:14:29.082074 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.082077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.082096 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.082196 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:29.082744 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.083752 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.084946 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.085232 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.085287 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.087539 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:29.087547 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:29.087547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:29.087555 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:29.089085 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.089500 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.089502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.089503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.089564 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.089786 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.089787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.089787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.089930 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:29.090021 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.090221 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.090315 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":74,"threads_started":1}
I20260808 07:14:29.090811 8170 tablet_service.cc:1514] Processing DeleteTablet for tablet 20e974de79684426b576f69db380646b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:08 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:45710
I20260808 07:14:29.091358 8255 tablet_replica.cc:330] stopping tablet replica
I20260808 07:14:29.091472 8255 raft_consensus.cc:2229] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:29.091543 8255 raft_consensus.cc:2258] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:29.091922 8255 ts_tablet_manager.cc:1905] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:29.094532 8255 ts_tablet_manager.cc:1918] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:29.094610 8255 log.cc:1196] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal/wals/20e974de79684426b576f69db380646b
I20260808 07:14:29.094843 8255 ts_tablet_manager.cc:1939] T 20e974de79684426b576f69db380646b P 2f5393c2f0cf44648d76427d1c65f20c: Deleting consensus metadata
I20260808 07:14:29.095357 8014 catalog_manager.cc:4841] TS 2f5393c2f0cf44648d76427d1c65f20c (127.22.2.129:46653): tablet 20e974de79684426b576f69db380646b (table test-table [id=e2f58cbe71c54e5cbd55a9dab9352aab]) successfully deleted
W20260808 07:14:29.163112 8252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:29.163321 8252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:29.163347 8252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:29.164561 8252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:29.164604 8252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:29.164631 8252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:14:29.166112 8252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:35669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=34001
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:29.166359 8252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:29.166622 8252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:29.171824 8264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:29.171839 8261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:29.171989 8252 server_base.cc:1034] running on GCE node
W20260808 07:14:29.171834 8262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:29.172196 8252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:29.172412 8252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:29.173588 8252 hybrid_clock.cc:648] HybridClock initialized: now 1786173269173577 us; error 27 us; skew 500 ppm
Aug 08 07:14:29 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173269, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:29.176031 8252 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:14:29.176476 8252 webserver.cc:460] Webserver started at http://127.22.2.130:34001/ using document root <none> and password file <none>
I20260808 07:14:29.176705 8252 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:29.176782 8252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:29.178159 8252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:29.178880 8272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:29.179049 8252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:29.179138 8252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "e6748dc79f1448af871d9d035a40d24f"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "603ab1a26b87b6c81c3f072d3116458a"
server_key_iv: "404ce9518deedd96e12428565db4939c"
server_key_version: "encryptionkey@0"
I20260808 07:14:29.179489 8252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:29.201275 8252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:29.203649 8252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:29.203771 8252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:29.203966 8252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:29.204424 8280 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:14:29.205878 8252 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:14:29.205924 8252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:14:29.205950 8252 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:14:29.206573 8252 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:14:29.206607 8252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:14:29.206713 8280 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Bootstrap starting.
I20260808 07:14:29.208525 8280 log.cc:824] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:29.211544 8280 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:14:29.211891 8280 tablet_bootstrap.cc:492] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Bootstrap complete.
I20260808 07:14:29.212046 8280 ts_tablet_manager.cc:1397] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 07:14:29.212741 8252 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35669
I20260808 07:14:29.212818 8403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35669 every 8 connection(s)
I20260808 07:14:29.213168 8252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:14:29.213830 8280 raft_consensus.cc:348] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.214181 8280 raft_consensus.cc:729] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6748dc79f1448af871d9d035a40d24f, State: Initialized, Role: FOLLOWER
I20260808 07:14:29.214320 8280 consensus_queue.cc:260] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.214593 8280 ts_tablet_manager.cc:1428] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Aug 08 07:14:29 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173269, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:14:29.222172 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 8252
I20260808 07:14:29.222501 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:38609
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:35535
--webserver_port=39907
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:14:29.223006 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.213944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:34471 (local address 127.22.2.190:35535)
0808 07:14:29.214019 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.214022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.214243 (+ 221us) server_negotiation.cc:407] Connection header received
0808 07:14:29.214457 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.214459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.214470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.214542 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:29.215182 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.215764 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.216724 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.216921 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.216975 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.219972 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:29.219974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:29.219974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:29.219981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:29.221489 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.221906 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.221907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.221908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.221985 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.222302 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.222303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.222304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.222427 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:29.222485 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.222850 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.222938 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:14:29.223265 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.213898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:34471)
0808 07:14:29.214096 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.214148 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:29.214340 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:29.214597 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:29.214601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:29.214919 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:29.215101 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.215106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.215880 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.215881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.216653 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.216655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.217034 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.217035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.217166 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.217731 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:29.217740 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:29.219882 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:29.221576 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:29.221582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:29.221584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:29.221820 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:29.222062 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:29.222063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:29.222063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:29.222208 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:29.222501 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:29.222508 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:29.222674 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 07:14:29.222820 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 07:14:29.222995 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260808 07:14:29.223960 8404 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35535
I20260808 07:14:29.224056 8404 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:29.224287 8404 heartbeater.cc:511] Master 127.22.2.190:35535 requested a full tablet report, sending...
I20260808 07:14:29.224911 8029 ts_manager.cc:194] Registered new tserver with Master: e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
I20260808 07:14:29.225564 8029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:34471
I20260808 07:14:29.226521 8404 heartbeater.cc:503] Master 127.22.2.190:35535 was elected leader, sending a full tablet report...
I20260808 07:14:29.230706 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.225288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.22.2.130:35669)
0808 07:14:29.227658 (+ 2370us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.227662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.227671 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:29.227698 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.227699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.227711 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.227799 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:29.227958 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.228744 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.229803 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.230296 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.230386 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.230441 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.230591 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.230630 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2336,"thread_start_us":85,"threads_started":1}
I20260808 07:14:29.230695 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.225232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35669 (local address 127.0.0.1:54642)
0808 07:14:29.225690 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.225701 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:29.225742 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:29.227828 (+ 2086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:29.227829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:29.227831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:29.227877 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.227879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.228829 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.228830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.229701 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.229703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.230384 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.230385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.230460 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.230491 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 07:14:29.230573 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:14:29.230623 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442}
I20260808 07:14:29.231310 8328 tablet_service.cc:1514] Processing DeleteTablet for tablet 20e974de79684426b576f69db380646b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:08 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:54642
I20260808 07:14:29.232115 8411 tablet_replica.cc:330] stopping tablet replica
I20260808 07:14:29.232235 8411 raft_consensus.cc:2229] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:29.232324 8411 raft_consensus.cc:2258] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:29.232645 8411 ts_tablet_manager.cc:1905] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:29.234954 8411 ts_tablet_manager.cc:1918] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:29.235025 8411 log.cc:1196] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/20e974de79684426b576f69db380646b
I20260808 07:14:29.235263 8411 ts_tablet_manager.cc:1939] T 20e974de79684426b576f69db380646b P e6748dc79f1448af871d9d035a40d24f: Deleting consensus metadata
I20260808 07:14:29.235831 8016 catalog_manager.cc:4841] TS e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669): tablet 20e974de79684426b576f69db380646b (table test-table [id=e2f58cbe71c54e5cbd55a9dab9352aab]) successfully deleted
W20260808 07:14:29.307044 8409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:29.307293 8409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:29.307333 8409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:29.308513 8409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:29.308583 8409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:29.308625 8409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:14:29.310072 8409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:38609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=39907
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:29.310348 8409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:29.310583 8409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:29.316171 8417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:29.316159 8416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:29.316159 8419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:29.316453 8409 server_base.cc:1034] running on GCE node
I20260808 07:14:29.316655 8409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:29.316915 8409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:29.318085 8409 hybrid_clock.cc:648] HybridClock initialized: now 1786173269318069 us; error 26 us; skew 500 ppm
Aug 08 07:14:29 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173269, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:29.320688 8409 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:14:29.321118 8409 webserver.cc:460] Webserver started at http://127.22.2.131:39907/ using document root <none> and password file <none>
I20260808 07:14:29.321358 8409 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:29.321445 8409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:29.322783 8409 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:29.323366 8426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:29.323542 8409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:29.323684 8409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "0844581a67c74792a4d7312c06dc2fa3"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "764f9cdf369da3aaa549d19404c7a7d1"
server_key_iv: "fd47ceb1d55305177c07300b2237e50c"
server_key_version: "encryptionkey@0"
I20260808 07:14:29.324054 8409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:29.338179 8409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:29.340190 8409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:29.340348 8409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:29.340567 8409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:29.340874 8409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:29.340927 8409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:29.340969 8409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:29.340997 8409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:29.347059 8409 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:38609
I20260808 07:14:29.347100 8553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:38609 every 8 connection(s)
I20260808 07:14:29.347469 8409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:29.348990 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 8409
Aug 08 07:14:29 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173269, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:14:29.355952 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.349391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.22.2.190:35535)
0808 07:14:29.349530 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.349532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.349626 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:14:29.349661 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.349662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.349675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.349754 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:29.350387 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.350875 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.351722 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.351935 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.352020 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.353264 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:29.353274 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:29.353274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:29.353281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:29.354704 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.355071 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.355073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.355073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.355134 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.355344 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.355346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.355346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.355462 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:29.355672 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.355711 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.355817 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260808 07:14:29.357512 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.348207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:34299 (local address 127.22.2.190:35535)
0808 07:14:29.348255 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.348257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.348485 (+ 228us) server_negotiation.cc:407] Connection header received
0808 07:14:29.348974 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.348976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.348989 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.349041 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:29.350199 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.350705 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.351783 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.351998 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.352077 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.354586 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:29.354588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:29.354588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:29.354595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:29.356246 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.356716 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.356718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.356718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.356780 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.357030 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.357031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.357031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.357129 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:29.357186 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.357393 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.357455 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:14:29.357712 8556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.348164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:34299)
0808 07:14:29.348354 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.348395 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:29.348865 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:29.349120 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:29.349124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:29.349887 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:29.350088 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.350093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.350805 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.350807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.351693 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.351696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.352088 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.352090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.352232 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.352799 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:29.352809 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:29.354486 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:29.356310 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:29.356318 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:29.356320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:29.356634 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:29.356857 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:29.356858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:29.356858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:29.356971 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:29.357180 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:29.357183 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:29.357286 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 07:14:29.357363 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:14:29.357476 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260808 07:14:29.358165 8554 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35535
I20260808 07:14:29.358243 8554 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:29.358449 8554 heartbeater.cc:511] Master 127.22.2.190:35535 requested a full tablet report, sending...
I20260808 07:14:29.358814 8029 ts_manager.cc:194] Registered new tserver with Master: 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609)
I20260808 07:14:29.359150 8029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:34299
I20260808 07:14:29.367796 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:14:29.370816 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.368239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.22.2.190:35535)
0808 07:14:29.368305 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.368307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.368431 (+ 124us) server_negotiation.cc:407] Connection header received
0808 07:14:29.368516 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.368517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.368571 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.368650 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:29.368792 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.369264 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.370028 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.370238 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.370328 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.370521 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:29.370576 (+ 55us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 07:14:29.370727 (+ 151us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32864: token expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 07:14:29.370898 8248 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32864: token expired
W20260808 07:14:29.370960 7946 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.2.190:35535)
I20260808 07:14:29.377231 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.371204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32872 (local address 127.22.2.190:35535)
0808 07:14:29.371268 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.371269 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.371279 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:14:29.371320 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.371321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.371330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.371380 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:29.371925 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.372375 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.373130 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.373319 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.373404 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.374649 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:29.374650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:29.374651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:29.374658 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:29.376025 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.376395 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.376397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.376397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.376460 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:29.376677 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:29.376678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:29.376678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:29.376791 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:29.376917 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.377049 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.377125 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:14:29.378397 8029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:14:29.380051 8029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:14:29.382130 8328 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7e5859d794498985d76e3d6b27e537 (DEFAULT_TABLE table=test-table [id=1b084c1777134a76be3ad79282650fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:29.382197 8170 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7e5859d794498985d76e3d6b27e537 (DEFAULT_TABLE table=test-table [id=1b084c1777134a76be3ad79282650fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:29.382359 8170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7e5859d794498985d76e3d6b27e537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:29.382356 8328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7e5859d794498985d76e3d6b27e537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:29.384075 8280 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: Bootstrap starting.
I20260808 07:14:29.384075 8123 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Bootstrap starting.
I20260808 07:14:29.384513 8123 tablet_bootstrap.cc:654] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:29.384522 8280 tablet_bootstrap.cc:654] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:29.385175 8123 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: No bootstrap required, opened a new log
I20260808 07:14:29.385229 8123 ts_tablet_manager.cc:1397] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:29.385576 8123 raft_consensus.cc:348] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.385667 8123 raft_consensus.cc:374] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:29.385725 8123 raft_consensus.cc:729] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5393c2f0cf44648d76427d1c65f20c, State: Initialized, Role: FOLLOWER
I20260808 07:14:29.385771 8123 consensus_queue.cc:260] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.385906 8123 ts_tablet_manager.cc:1428] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:14:29.386386 8280 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: No bootstrap required, opened a new log
I20260808 07:14:29.386441 8280 ts_tablet_manager.cc:1397] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:14:29.386588 8280 raft_consensus.cc:348] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.386644 8280 raft_consensus.cc:374] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:29.386687 8280 raft_consensus.cc:729] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6748dc79f1448af871d9d035a40d24f, State: Initialized, Role: FOLLOWER
I20260808 07:14:29.386732 8280 consensus_queue.cc:260] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.386809 8280 ts_tablet_manager.cc:1428] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:29.387934 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.381905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38609 (local address 127.0.0.1:50326)
0808 07:14:29.381946 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.381958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:29.381996 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:29.384488 (+ 2492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:29.384489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:29.384491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:29.384537 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.384539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.385690 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.385692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.386801 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.386804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.387683 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.387684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.387744 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.387747 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:29.387807 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:14:29.387864 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:14:29.387946 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.384106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.22.2.131:38609)
0808 07:14:29.384290 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.384295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.384305 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:14:29.384354 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.384356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.384369 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.384468 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:29.384596 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.385596 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.386913 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.387613 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.387685 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.387750 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.387824 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.387864 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":108,"threads_started":1}
I20260808 07:14:29.388626 8479 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7e5859d794498985d76e3d6b27e537 (DEFAULT_TABLE table=test-table [id=1b084c1777134a76be3ad79282650fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:29.388908 8479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7e5859d794498985d76e3d6b27e537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:29.391839 8564 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Bootstrap starting.
I20260808 07:14:29.392429 8564 tablet_bootstrap.cc:654] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:29.392974 8564 log.cc:824] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:29.393712 8564 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: No bootstrap required, opened a new log
I20260808 07:14:29.393790 8564 ts_tablet_manager.cc:1397] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:14:29.395226 8564 raft_consensus.cc:348] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.395373 8564 raft_consensus.cc:374] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:29.395396 8564 raft_consensus.cc:729] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0844581a67c74792a4d7312c06dc2fa3, State: Initialized, Role: FOLLOWER
I20260808 07:14:29.395537 8564 consensus_queue.cc:260] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.395855 8564 ts_tablet_manager.cc:1428] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:14:29.395910 8554 heartbeater.cc:503] Master 127.22.2.190:35535 was elected leader, sending a full tablet report...
W20260808 07:14:29.463726 8405 tablet.cc:2369] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:14:29.565410 8246 tablet.cc:2369] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:14:29.597574 8251 raft_consensus.cc:482] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:29.597725 8251 raft_consensus.cc:504] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
W20260808 07:14:29.598088 8555 tablet.cc:2369] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:14:29.598194 8251 leader_election.cc:288] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609), e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
I20260808 07:14:29.601383 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.598521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:40775 (local address 127.22.2.131:38609)
0808 07:14:29.598583 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.598586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.598653 (+ 67us) server_negotiation.cc:407] Connection header received
0808 07:14:29.598785 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.598786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.598797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.598876 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:29.599037 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.599607 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.600610 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.600987 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.601085 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.601093 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.601266 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.601309 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:14:29.601400 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.598607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56849 (local address 127.22.2.130:35669)
0808 07:14:29.598669 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.598671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.598680 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:29.598760 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.598761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.598774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.598855 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:29.599059 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.599533 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.600601 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.600990 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.601076 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.601085 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.601270 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.601313 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:14:29.601413 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.598501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35669 (local address 127.22.2.129:56849)
0808 07:14:29.598620 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.598631 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:29.598692 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:29.598884 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:29.598885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:29.598887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:29.598975 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.598977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.599661 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.599662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.600506 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.600509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.601086 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.601088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.601169 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.601186 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 07:14:29.601261 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:14:29.601308 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":57,"thread_start_us":63,"threads_started":1}
I20260808 07:14:29.601428 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.598427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38609 (local address 127.22.2.129:40775)
0808 07:14:29.598578 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.598592 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:29.598666 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:29.598880 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:29.598882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:29.598885 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:29.598975 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.598981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.599676 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.599677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.600500 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:29.600503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.601089 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.601090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:29.601164 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.601182 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 07:14:29.601257 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:14:29.601313 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":17,"thread_start_us":67,"threads_started":1}
I20260808 07:14:29.601696 8352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7e5859d794498985d76e3d6b27e537" candidate_uuid: "2f5393c2f0cf44648d76427d1c65f20c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6748dc79f1448af871d9d035a40d24f" is_pre_election: true
I20260808 07:14:29.601852 8352 raft_consensus.cc:2454] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f5393c2f0cf44648d76427d1c65f20c in term 0.
I20260808 07:14:29.601913 8502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7e5859d794498985d76e3d6b27e537" candidate_uuid: "2f5393c2f0cf44648d76427d1c65f20c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0844581a67c74792a4d7312c06dc2fa3" is_pre_election: true
I20260808 07:14:29.602056 8502 raft_consensus.cc:2454] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f5393c2f0cf44648d76427d1c65f20c in term 0.
I20260808 07:14:29.602087 8120 leader_election.cc:302] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f5393c2f0cf44648d76427d1c65f20c, e6748dc79f1448af871d9d035a40d24f; no voters:
I20260808 07:14:29.602187 8251 raft_consensus.cc:2781] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:14:29.602257 8251 raft_consensus.cc:482] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:14:29.602283 8251 raft_consensus.cc:3037] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:29.603006 8251 raft_consensus.cc:504] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.603160 8251 leader_election.cc:288] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 1 election: Requested vote from peers 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609), e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
I20260808 07:14:29.603320 8352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7e5859d794498985d76e3d6b27e537" candidate_uuid: "2f5393c2f0cf44648d76427d1c65f20c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6748dc79f1448af871d9d035a40d24f"
I20260808 07:14:29.603343 8502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7e5859d794498985d76e3d6b27e537" candidate_uuid: "2f5393c2f0cf44648d76427d1c65f20c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0844581a67c74792a4d7312c06dc2fa3"
I20260808 07:14:29.603421 8352 raft_consensus.cc:3037] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:29.603443 8502 raft_consensus.cc:3037] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:29.604202 8502 raft_consensus.cc:2454] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f5393c2f0cf44648d76427d1c65f20c in term 1.
I20260808 07:14:29.604417 8118 leader_election.cc:302] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0844581a67c74792a4d7312c06dc2fa3, 2f5393c2f0cf44648d76427d1c65f20c; no voters:
I20260808 07:14:29.604578 8251 raft_consensus.cc:2781] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:29.604712 8251 raft_consensus.cc:686] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 LEADER]: Becoming Leader. State: Replica: 2f5393c2f0cf44648d76427d1c65f20c, State: Running, Role: LEADER
I20260808 07:14:29.604792 8251 consensus_queue.cc:237] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:29.604871 8352 raft_consensus.cc:2454] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f5393c2f0cf44648d76427d1c65f20c in term 1.
I20260808 07:14:29.605533 8029 catalog_manager.cc:5495] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f5393c2f0cf44648d76427d1c65f20c (127.22.2.129). New cstate: current_term: 1 leader_uuid: "2f5393c2f0cf44648d76427d1c65f20c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } health_report { overall_health: HEALTHY } } }
I20260808 07:14:29.612294 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.609763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.22.2.129:46653)
0808 07:14:29.609941 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:29.609943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:29.609950 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:14:29.609979 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:29.609979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:29.609993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:29.610078 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:29.610190 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.610664 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.611415 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:29.611677 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:29.611751 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:29.611922 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:29.612029 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:14:29.612107 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:29.612162 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 07:14:29.612183 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260808 07:14:29.613723 8352 raft_consensus.cc:1261] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Refusing update from remote peer 2f5393c2f0cf44648d76427d1c65f20c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:29.613801 8502 raft_consensus.cc:1261] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 1 FOLLOWER]: Refusing update from remote peer 2f5393c2f0cf44648d76427d1c65f20c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:29.614077 8570 consensus_queue.cc:1035] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:29.614203 8251 consensus_queue.cc:1035] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:29.616681 8577 mvcc.cc:205] Tried to move back new op lower bound from 7316165712335433728 to 7316165712301891584. Current Snapshot: MvccSnapshot[applied={T|T < 7316165712335433728}]
I20260808 07:14:29.618360 8028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:29.618460 8028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:29.619297 8028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Sending DeleteTablet for 3 replicas of tablet 3e7e5859d794498985d76e3d6b27e537
I20260808 07:14:29.619628 8170 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7e5859d794498985d76e3d6b27e537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:29 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:45710
I20260808 07:14:29.619693 8328 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7e5859d794498985d76e3d6b27e537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:29 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:54642
I20260808 07:14:29.619788 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 8075
I20260808 07:14:29.619812 8411 tablet_replica.cc:330] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: stopping tablet replica
I20260808 07:14:29.619843 8479 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7e5859d794498985d76e3d6b27e537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:29 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:50326
I20260808 07:14:29.619875 8411 raft_consensus.cc:2229] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:29.619925 8411 raft_consensus.cc:2258] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:29.621666 8585 tablet_replica.cc:330] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: stopping tablet replica
I20260808 07:14:29.621711 8411 ts_tablet_manager.cc:1905] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:29.621757 8585 raft_consensus.cc:2229] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:29.621836 8585 raft_consensus.cc:2258] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:29.622233 8585 ts_tablet_manager.cc:1905] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:29.623764 8411 ts_tablet_manager.cc:1918] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:29.623821 8411 log.cc:1196] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/3e7e5859d794498985d76e3d6b27e537
I20260808 07:14:29.624001 8411 ts_tablet_manager.cc:1939] T 3e7e5859d794498985d76e3d6b27e537 P e6748dc79f1448af871d9d035a40d24f: Deleting consensus metadata
I20260808 07:14:29.624375 8016 catalog_manager.cc:4841] TS e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669): tablet 3e7e5859d794498985d76e3d6b27e537 (table test-table [id=1b084c1777134a76be3ad79282650fe7]) successfully deleted
I20260808 07:14:29.624758 8585 ts_tablet_manager.cc:1918] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:29.624840 8585 log.cc:1196] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/3e7e5859d794498985d76e3d6b27e537
I20260808 07:14:29.625144 8585 ts_tablet_manager.cc:1939] T 3e7e5859d794498985d76e3d6b27e537 P 0844581a67c74792a4d7312c06dc2fa3: Deleting consensus metadata
I20260808 07:14:29.625576 8014 catalog_manager.cc:4841] TS 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609): tablet 3e7e5859d794498985d76e3d6b27e537 (table test-table [id=1b084c1777134a76be3ad79282650fe7]) successfully deleted
W20260808 07:14:29.628861 8014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:46653 (error 108)
I20260808 07:14:29.629261 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 8252
I20260808 07:14:29.629393 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.629230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:46653 (local address 127.0.0.1:45738)
0808 07:14:29.629298 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.629354 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:46653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 07:14:29.630523 8014 catalog_manager.cc:4584] TS 2f5393c2f0cf44648d76427d1c65f20c (127.22.2.129:46653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e7e5859d794498985d76e3d6b27e537: Network error: Client connection negotiation failed: client connection to 127.22.2.129:46653: connect: Connection refused (error 111)
I20260808 07:14:29.635638 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 8409
I20260808 07:14:29.641304 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 7994
I20260808 07:14:49.648895 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35535
--webserver_interface=127.22.2.190
--webserver_port=42907
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:49.735373 8587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:49.735646 8587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:49.735674 8587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:49.736936 8587 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:14:49.736976 8587 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:14:49.736989 8587 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:14:49.737001 8587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:49.737047 8587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:49.737075 8587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:49.737099 8587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:49.737119 8587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:49.738703 8587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=42907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:49.738962 8587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:49.739218 8587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:49.744835 8592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:49.744804 8595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:49.744853 8587 server_base.cc:1034] running on GCE node
W20260808 07:14:49.744819 8593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:49.745379 8587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:49.745648 8587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:49.746790 8587 hybrid_clock.cc:648] HybridClock initialized: now 1786173289746778 us; error 27 us; skew 500 ppm
Aug 08 07:14:49 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:49.749396 8587 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:14:49.749886 8587 webserver.cc:460] Webserver started at http://127.22.2.190:42907/ using document root <none> and password file <none>
I20260808 07:14:49.750114 8587 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:49.750185 8587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:49.751649 8587 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:49.752421 8602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:49.752718 8587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:49.752825 8587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "d86e64699ea74c44b1bfcc01e1c6800c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "c0d01cea5fc9d65ee6edf5f6a756d567"
server_key_iv: "2ec53e95bda7b7c494c0dff85ccbc035"
server_key_version: "encryptionkey@0"
I20260808 07:14:49.753181 8587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:49.769272 8587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:49.771695 8587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:49.771865 8587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:49.776239 8587 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35535
I20260808 07:14:49.776281 8659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35535 every 8 connection(s)
I20260808 07:14:49.776814 8587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:49.778311 8660 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:14:49.778939 8660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Bootstrap starting.
I20260808 07:14:49.785674 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 8587
I20260808 07:14:49.786000 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:46653
--local_ip_for_outbound_sockets=127.22.2.129
--tserver_master_addrs=127.22.2.190:35535
--webserver_port=42675
--webserver_interface=127.22.2.129
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:14:49.788705 8660 log.cc:824] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:49.790980 8660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:14:49.791293 8660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Bootstrap complete.
I20260808 07:14:49.793836 8660 raft_consensus.cc:348] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:49.794087 8660 raft_consensus.cc:729] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86e64699ea74c44b1bfcc01e1c6800c, State: Initialized, Role: FOLLOWER
I20260808 07:14:49.794272 8660 consensus_queue.cc:260] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:49.794353 8660 raft_consensus.cc:388] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:49.794407 8660 raft_consensus.cc:482] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:49.794454 8660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:14:49.795450 8660 raft_consensus.cc:504] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:49.795591 8660 leader_election.cc:302] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d86e64699ea74c44b1bfcc01e1c6800c; no voters:
I20260808 07:14:49.795812 8660 leader_election.cc:288] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:14:49.795853 8665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:14:49.795976 8665 raft_consensus.cc:686] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [term 3 LEADER]: Becoming Leader. State: Replica: d86e64699ea74c44b1bfcc01e1c6800c, State: Running, Role: LEADER
I20260808 07:14:49.796073 8665 consensus_queue.cc:237] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } }
I20260808 07:14:49.796097 8660 sys_catalog.cc:564] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:49.796367 8665 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d86e64699ea74c44b1bfcc01e1c6800c. Latest consensus state: current_term: 3 leader_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } } }
I20260808 07:14:49.796443 8665 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:49.796372 8666 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d86e64699ea74c44b1bfcc01e1c6800c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35535 } } }
I20260808 07:14:49.796499 8666 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:49.796754 8670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:49.797652 8670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:49.797755 8670 catalog_manager.cc:1259] Loaded cluster ID: 4d158b476de84bf583674b4ff8a312a7
I20260808 07:14:49.797781 8670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:49.798936 8670 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:49.799126 8670 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Loaded TSK: 0
I20260808 07:14:49.799180 8670 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d86e64699ea74c44b1bfcc01e1c6800c: Loaded TSK: 1
F20260808 07:14:49.799413 8670 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0399b109ad google::LogMessage::Fail() at ??:0
@ 0x7f0399b1388a google::LogMessage::SendToLog() at ??:0
@ 0x7f0399b104de google::LogMessage::Flush() at ??:0
@ 0x7f0399b11a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f039c27d104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f039c27ad41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f03a1a6a1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f03a1a6a48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f03a1a6a594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f03a1a30c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f03a1a3aad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f039b9de0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f039b9d3954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f039c8376db start_thread at ??:0
@ 0x7f039860e61f clone at ??:0
W20260808 07:14:49.886559 8662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:49.886797 8662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:49.886838 8662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:49.888065 8662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:49.888132 8662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:49.888177 8662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:14:49.889662 8662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:46653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=42675
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:49.889957 8662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:49.890198 8662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:49.896550 8707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:49.896664 8706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:49.896600 8709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:49.897212 8662 server_base.cc:1034] running on GCE node
I20260808 07:14:49.897449 8662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:49.897698 8662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:49.898904 8662 hybrid_clock.cc:648] HybridClock initialized: now 1786173289898885 us; error 37 us; skew 500 ppm
Aug 08 07:14:49 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:49.901618 8662 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:14:49.902083 8662 webserver.cc:460] Webserver started at http://127.22.2.129:42675/ using document root <none> and password file <none>
I20260808 07:14:49.902326 8662 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:49.902398 8662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:49.903842 8662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:14:49.904655 8718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:49.904855 8662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:49.904965 8662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "2f5393c2f0cf44648d76427d1c65f20c"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "da5349e2ac2a26b1f7cd06703cc0d5c7"
server_key_iv: "c115102b20712a9b9322321ab9dd6e96"
server_key_version: "encryptionkey@0"
I20260808 07:14:49.905359 8662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:49.924762 8662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:49.927476 8662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:49.927703 8662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:49.928009 8662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:49.928596 8725 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:14:49.931003 8662 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:14:49.931097 8662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 07:14:49.931152 8662 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:14:49.932119 8662 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:14:49.932184 8662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:49.932255 8725 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Bootstrap starting.
I20260808 07:14:49.940866 8662 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:46653
I20260808 07:14:49.941031 8844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:46653 every 8 connection(s)
I20260808 07:14:49.941354 8662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:49.942059 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 8662
I20260808 07:14:49.942442 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:35669
--local_ip_for_outbound_sockets=127.22.2.130
--tserver_master_addrs=127.22.2.190:35535
--webserver_port=34001
--webserver_interface=127.22.2.130
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:14:49.944975 8725 log.cc:824] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:49.947899 8725 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:14:49.948235 8725 tablet_bootstrap.cc:492] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Bootstrap complete.
I20260808 07:14:49.949630 8725 ts_tablet_manager.cc:1397] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.010s
I20260808 07:14:49.951095 8725 raft_consensus.cc:348] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:49.951447 8725 raft_consensus.cc:729] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5393c2f0cf44648d76427d1c65f20c, State: Initialized, Role: FOLLOWER
I20260808 07:14:49.951643 8725 consensus_queue.cc:260] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:49.951995 8725 ts_tablet_manager.cc:1428] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
W20260808 07:14:50.016148 8847 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:14:49.942150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:59457)
0808 07:14:49.942389 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:49.942445 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:49.943514 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:50.015930 (+ 72416us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":123,"threads_started":1}
W20260808 07:14:50.016474 8845 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:35535 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:35535: Client connection negotiation failed: client connection to 127.22.2.190:35535: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:14:50.016877 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.016723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:49911)
0808 07:14:50.016785 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:50.016841 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:14:50.017258 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.017142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:35669)
0808 07:14:50.017194 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:50.017225 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 07:14:50.017387 8845 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:14:50.031980 8848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:50.032179 8848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:50.032202 8848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:50.033358 8848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:50.033401 8848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:50.033452 8848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:14:50.034883 8848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:35669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=34001
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:50.035120 8848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:50.035349 8848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:50.041014 8857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:50.041024 8859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:50.041044 8856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:50.041309 8848 server_base.cc:1034] running on GCE node
I20260808 07:14:50.041517 8848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:50.041754 8848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:50.042920 8848 hybrid_clock.cc:648] HybridClock initialized: now 1786173290042907 us; error 29 us; skew 500 ppm
Aug 08 07:14:50 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173290, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:50.045482 8848 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:14:50.045920 8848 webserver.cc:460] Webserver started at http://127.22.2.130:34001/ using document root <none> and password file <none>
I20260808 07:14:50.046146 8848 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:50.046226 8848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:50.047549 8848 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:50.048280 8866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:50.048470 8848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:50.048569 8848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "e6748dc79f1448af871d9d035a40d24f"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "603ab1a26b87b6c81c3f072d3116458a"
server_key_iv: "404ce9518deedd96e12428565db4939c"
server_key_version: "encryptionkey@0"
I20260808 07:14:50.048930 8848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:50.063625 8848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:50.065972 8848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:50.066139 8848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:50.066404 8848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:50.066720 8848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:50.066771 8848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:50.066825 8848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:50.066855 8848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:50.072885 8848 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35669
I20260808 07:14:50.072942 8993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35669 every 8 connection(s)
I20260808 07:14:50.073259 8848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:14:50.074294 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.073928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:36625)
0808 07:14:50.074074 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:50.074144 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
W20260808 07:14:50.074702 8994 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:35535 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:35535: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
I20260808 07:14:50.074962 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.074868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:35321)
0808 07:14:50.074917 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:50.074943 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:14:50.075237 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.075165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:50027)
0808 07:14:50.075202 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:50.075223 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 07:14:50.075364 8994 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:14:50.083148 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 8848
I20260808 07:14:50.083447 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:38609
--local_ip_for_outbound_sockets=127.22.2.131
--tserver_master_addrs=127.22.2.190:35535
--webserver_port=39907
--webserver_interface=127.22.2.131
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:50.165934 8998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:50.166131 8998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:50.166155 8998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:50.167320 8998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:50.167364 8998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:50.167425 8998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:14:50.168938 8998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:38609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=39907
--tserver_master_addrs=127.22.2.190:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:14:50.169176 8998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:50.169420 8998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:50.174765 9004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:50.174782 9003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:50.174998 8998 server_base.cc:1034] running on GCE node
W20260808 07:14:50.175144 9007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:50.175341 8998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:50.175666 8998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:50.176776 8998 hybrid_clock.cc:648] HybridClock initialized: now 1786173290176774 us; error 40 us; skew 500 ppm
Aug 08 07:14:50 dist-test-slave-6k22 krb5kdc[7375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173290, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:50.180433 8998 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:14:50.181016 8998 webserver.cc:460] Webserver started at http://127.22.2.131:39907/ using document root <none> and password file <none>
I20260808 07:14:50.181318 8998 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:50.181391 8998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:50.183238 8998 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:50.183933 9014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:50.184108 8998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:50.184197 8998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "0844581a67c74792a4d7312c06dc2fa3"
format_stamp: "Formatted at 2026-08-08 07:14:08 on dist-test-slave-6k22"
server_key: "764f9cdf369da3aaa549d19404c7a7d1"
server_key_iv: "fd47ceb1d55305177c07300b2237e50c"
server_key_version: "encryptionkey@0"
I20260808 07:14:50.184567 8998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:50.202451 8998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:50.204424 8998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:50.204593 8998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:50.204852 8998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:50.205164 8998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:50.205219 8998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:50.205255 8998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:50.205308 8998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:50.211790 8998 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:38609
I20260808 07:14:50.211853 9136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:38609 every 8 connection(s)
I20260808 07:14:50.212219 8998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:50.213371 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.212960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:35433)
0808 07:14:50.213147 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:50.213225 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
W20260808 07:14:50.213863 9137 heartbeater.cc:644] Failed to heartbeat to 127.22.2.190:35535 (0 consecutive failures): Network error: Failed to ping master at 127.22.2.190:35535: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
I20260808 07:14:50.214198 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.214060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:57067)
0808 07:14:50.214130 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:50.214172 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 07:14:50.214524 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.214433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:41071)
0808 07:14:50.214478 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:50.214505 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:14:50.214725 9137 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:14:50.219113 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 8998
I20260808 07:14:51.018132 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.017868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:35227)
0808 07:14:51.018034 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:51.018080 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
I20260808 07:14:51.075919 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.075716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:46625)
0808 07:14:51.075840 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:51.075870 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260808 07:14:51.215325 9143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.215080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:46941)
0808 07:14:51.215234 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:51.215272 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260808 07:14:51.496641 9144 raft_consensus.cc:482] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:51.496763 9144 raft_consensus.cc:504] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:51.497062 9144 leader_election.cc:288] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609), e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
Aug 08 07:14:51 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.131@KRBTEST.COM
Aug 08 07:14:51 dist-test-slave-6k22 krb5kdc[7375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.130@KRBTEST.COM
I20260808 07:14:51.510015 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.497432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:42021 (local address 127.22.2.131:38609)
0808 07:14:51.497571 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:51.497579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:51.497607 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:14:51.498209 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:51.498212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:51.498275 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:51.498447 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:51.499484 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:51.500935 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:51.502421 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:51.503050 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:51.503166 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:51.505968 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:51.505972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:51.505972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:51.505982 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:51.508103 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:51.508714 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:51.508718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:51.508718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:51.508818 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:51.509087 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:51.509089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:51.509089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:51.509395 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:51.509524 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:51.509761 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 07:14:51.509890 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":48,"threads_started":1}
I20260808 07:14:51.510003 9145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.497236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38609 (local address 127.22.2.129:42021)
0808 07:14:51.497409 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:51.497420 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:51.497480 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:51.498527 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:51.498530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:51.499054 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:51.499345 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:51.499353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:51.501077 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:51.501078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:51.502325 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:51.502328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:51.503157 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:51.503157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:51.503288 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:51.503976 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:51.503984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:51.505809 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:51.508186 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:51.508209 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:51.508212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:51.508628 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:51.508899 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:51.508900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:51.508900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:51.509019 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:51.509494 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:51.509495 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:51.509580 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 07:14:51.509692 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:14:51.509903 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":95,"thread_start_us":77,"threads_started":1}
I20260808 07:14:51.510036 9146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.497383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:44599 (local address 127.22.2.130:35669)
0808 07:14:51.497553 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:51.497566 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:51.497599 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:14:51.498254 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:51.498257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:51.498296 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:51.498447 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:51.499485 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:51.500935 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:51.502422 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:51.503050 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:51.503166 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:51.506203 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:51.506207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:51.506208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:51.506220 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:51.508167 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:51.508710 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:51.508713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:51.508714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:51.508812 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:51.509086 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:51.509087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:51.509087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:51.509344 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:51.509432 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:51.509759 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 07:14:51.509892 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260808 07:14:51.510864 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.497248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35669 (local address 127.22.2.129:44599)
0808 07:14:51.497337 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:51.497351 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:51.497423 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:51.498527 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:51.498530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:51.499054 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:51.499345 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:51.499353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:51.501058 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:51.501059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:51.502325 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:51.502328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:51.503155 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:51.503157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:51.503288 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:51.503971 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:51.503982 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:51.506105 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:51.508256 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:51.508257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:51.508257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:51.508627 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:51.508893 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:51.508894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:51.508894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:51.509019 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:51.509425 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:51.509431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:51.509579 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 07:14:51.509694 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:14:51.509990 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":165}
W20260808 07:14:51.510946 8720 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
W20260808 07:14:51.511549 8722 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
I20260808 07:14:51.511631 8722 leader_election.cc:302] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f5393c2f0cf44648d76427d1c65f20c; no voters: 0844581a67c74792a4d7312c06dc2fa3, e6748dc79f1448af871d9d035a40d24f
I20260808 07:14:51.511799 9144 raft_consensus.cc:2726] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:52.018970 9148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:52.018744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:54343)
0808 07:14:52.018881 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:52.018920 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260808 07:14:52.076648 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:52.076434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:57585)
0808 07:14:52.076567 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:52.076608 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260808 07:14:52.215987 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:52.215787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:47559)
0808 07:14:52.215910 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:52.215946 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260808 07:14:53.019793 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:53.019486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:34599)
0808 07:14:53.019696 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:53.019742 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":124,"threads_started":1}
I20260808 07:14:53.077312 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:53.077081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:37963)
0808 07:14:53.077220 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:53.077260 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260808 07:14:53.216809 9153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:53.216539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:37129)
0808 07:14:53.216734 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:53.216766 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":134,"threads_started":1}
I20260808 07:14:53.432089 9154 raft_consensus.cc:482] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:53.432193 9154 raft_consensus.cc:504] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:53.432384 9154 leader_election.cc:288] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609), e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
W20260808 07:14:53.432838 8722 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
W20260808 07:14:53.432925 8720 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
I20260808 07:14:53.433012 8720 leader_election.cc:302] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f5393c2f0cf44648d76427d1c65f20c; no voters: 0844581a67c74792a4d7312c06dc2fa3, e6748dc79f1448af871d9d035a40d24f
I20260808 07:14:53.433089 9154 raft_consensus.cc:2726] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:54.020581 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:54.020321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:54863)
0808 07:14:54.020486 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:54.020529 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":70,"threads_started":1}
I20260808 07:14:54.077951 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:54.077743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:56111)
0808 07:14:54.077879 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:54.077911 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260808 07:14:54.217545 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:54.217321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:36889)
0808 07:14:54.217463 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:54.217500 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260808 07:14:55.021394 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:55.021133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:58061)
0808 07:14:55.021297 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:55.021342 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260808 07:14:55.078610 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:55.078403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:43019)
0808 07:14:55.078534 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:55.078568 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260808 07:14:55.218253 9161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:55.218040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:35535)
0808 07:14:55.218172 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:55.218210 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260808 07:14:55.343523 9162 raft_consensus.cc:482] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:55.343678 9162 raft_consensus.cc:504] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:55.343924 9162 leader_election.cc:288] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609), e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
W20260808 07:14:55.344300 8722 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
W20260808 07:14:55.344425 8720 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
I20260808 07:14:55.344513 8720 leader_election.cc:302] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f5393c2f0cf44648d76427d1c65f20c; no voters: 0844581a67c74792a4d7312c06dc2fa3, e6748dc79f1448af871d9d035a40d24f
I20260808 07:14:55.344615 9162 raft_consensus.cc:2726] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:56.022193 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:56.021927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:43341)
0808 07:14:56.022092 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:56.022144 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260808 07:14:56.079342 9164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:56.079087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:37191)
0808 07:14:56.079258 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:56.079297 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260808 07:14:56.218961 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:56.218744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:57959)
0808 07:14:56.218880 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:56.218917 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260808 07:14:57.022960 9166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:57.022715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:59195)
0808 07:14:57.022872 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:57.022911 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260808 07:14:57.080050 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:57.079836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:35027)
0808 07:14:57.079968 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:57.080005 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260808 07:14:57.219900 9168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:57.219548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:41823)
0808 07:14:57.219763 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:57.219839 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":68,"threads_started":1}
I20260808 07:14:58.023738 9169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:58.023433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:40213)
0808 07:14:58.023649 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:58.023690 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
I20260808 07:14:58.080870 9170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:58.080573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:38223)
0808 07:14:58.080785 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:58.080824 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":142,"threads_started":1}
I20260808 07:14:58.220598 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:58.220397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:36877)
0808 07:14:58.220521 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:58.220553 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260808 07:14:59.024547 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:59.024300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:43969)
0808 07:14:59.024457 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:59.024495 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260808 07:14:59.081581 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:59.081362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:53497)
0808 07:14:59.081495 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:59.081533 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260808 07:14:59.176939 9174 raft_consensus.cc:482] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:59.177039 9174 raft_consensus.cc:504] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:14:59.177438 9174 leader_election.cc:288] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609), e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
W20260808 07:14:59.177842 8720 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
W20260808 07:14:59.177925 8722 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
I20260808 07:14:59.177994 8722 leader_election.cc:302] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f5393c2f0cf44648d76427d1c65f20c; no voters: 0844581a67c74792a4d7312c06dc2fa3, e6748dc79f1448af871d9d035a40d24f
I20260808 07:14:59.178089 9174 raft_consensus.cc:2726] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:59.221283 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:59.221071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:38863)
0808 07:14:59.221203 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:59.221239 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260808 07:15:00.025573 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:00.025313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:57295)
0808 07:15:00.025472 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:00.025521 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260808 07:15:00.082338 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:00.082098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:47691)
0808 07:15:00.082251 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:00.082290 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
I20260808 07:15:00.222041 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:00.221805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:44885)
0808 07:15:00.221951 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:00.221991 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260808 07:15:01.026337 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:01.026088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:49959)
0808 07:15:01.026246 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:01.026287 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260808 07:15:01.083050 9180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:01.082821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:39091)
0808 07:15:01.082957 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:01.083004 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260808 07:15:01.222908 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:01.222619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:53923)
0808 07:15:01.222811 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:01.222857 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":67,"threads_started":1}
I20260808 07:15:02.027124 9182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:02.026883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:54245)
0808 07:15:02.027023 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:02.027073 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260808 07:15:02.083796 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:02.083489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:44121)
0808 07:15:02.083714 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:02.083748 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":115,"threads_started":1}
I20260808 07:15:02.223739 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:02.223461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:46581)
0808 07:15:02.223634 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:02.223687 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260808 07:15:03.027966 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:03.027745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:45051)
0808 07:15:03.027877 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:03.027915 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260808 07:15:03.084506 9188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:03.084290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:33641)
0808 07:15:03.084420 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:03.084457 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260808 07:15:03.224498 9189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:03.224271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:56815)
0808 07:15:03.224412 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:03.224450 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260808 07:15:03.982025 9190 raft_consensus.cc:482] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:15:03.982175 9190 raft_consensus.cc:504] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0844581a67c74792a4d7312c06dc2fa3" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38609 } } peers { permanent_uuid: "e6748dc79f1448af871d9d035a40d24f" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35669 } } peers { permanent_uuid: "2f5393c2f0cf44648d76427d1c65f20c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 46653 } }
I20260808 07:15:03.982415 9190 leader_election.cc:288] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609), e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669)
W20260808 07:15:03.982813 8722 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6748dc79f1448af871d9d035a40d24f (127.22.2.130:35669): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
W20260808 07:15:03.982893 8720 leader_election.cc:341] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0844581a67c74792a4d7312c06dc2fa3 (127.22.2.131:38609): Not found: Tablet not found: 3e7e5859d794498985d76e3d6b27e537
I20260808 07:15:03.982933 8720 leader_election.cc:302] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f5393c2f0cf44648d76427d1c65f20c; no voters: 0844581a67c74792a4d7312c06dc2fa3, e6748dc79f1448af871d9d035a40d24f
I20260808 07:15:03.983024 9190 raft_consensus.cc:2726] T 3e7e5859d794498985d76e3d6b27e537 P 2f5393c2f0cf44648d76427d1c65f20c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:15:04.028648 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:04.028427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:39765)
0808 07:15:04.028559 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:04.028599 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260808 07:15:04.085196 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:04.084971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:37031)
0808 07:15:04.085110 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:04.085151 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260808 07:15:04.225200 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:04.224985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:34755)
0808 07:15:04.225119 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:04.225152 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260808 07:15:05.029438 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:05.029223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.129:44823)
0808 07:15:05.029353 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:05.029391 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260808 07:15:05.085911 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:05.085675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.130:56225)
0808 07:15:05.085830 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:05.085864 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260808 07:15:05.220975 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 8662
I20260808 07:15:05.225966 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:05.225765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35535 (local address 127.22.2.131:55615)
0808 07:15:05.225888 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:05.225923 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.190:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260808 07:15:05.228467 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 8848
I20260808 07:15:05.234831 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 8998
2026-08-08T07:15:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58528 ms)
[----------] 30 tests from SecurityITest (236500 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:05 dist-test-slave-6k22 krb5kdc[9202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:05 dist-test-slave-6k22 krb5kdc[9202](info): set up 2 sockets
Aug 08 07:15:05 dist-test-slave-6k22 krb5kdc[9202](info): commencing operation
krb5kdc: starting...
W20260808 07:15:06.365006 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:06 dist-test-slave-6k22 krb5kdc[9202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:06.492677 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41545
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:06.573999 9215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:06.574196 9215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:06.574218 9215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:06.575281 9215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:06.575313 9215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:06.575328 9215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:06.575340 9215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:06.575352 9215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:06.576790 9215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:41545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:06.577039 9215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:06.577244 9215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:06.582521 9221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:06.582499 9215 server_base.cc:1034] running on GCE node
W20260808 07:15:06.582499 9220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:06.582515 9223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:06.583091 9215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:06.583401 9215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:06.584549 9215 hybrid_clock.cc:648] HybridClock initialized: now 1786173306584528 us; error 47 us; skew 500 ppm
Aug 08 07:15:06 dist-test-slave-6k22 krb5kdc[9202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:06.586997 9215 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:06.587467 9215 webserver.cc:460] Webserver started at http://127.22.2.190:34565/ using document root <none> and password file <none>
I20260808 07:15:06.587745 9215 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:06.587826 9215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:06.587975 9215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:06.589025 9215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "8d155f5b703f4ff19ffc2ebeb3832b91"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "7c2173966c45a478722ad44e28dce24b"
server_key_iv: "01e5cb20ea3ff04d1dae88201e1c6aea"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.589419 9215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "8d155f5b703f4ff19ffc2ebeb3832b91"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "7c2173966c45a478722ad44e28dce24b"
server_key_iv: "01e5cb20ea3ff04d1dae88201e1c6aea"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.590723 9215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:06.591436 9232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:06.591663 9215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:06.591763 9215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "8d155f5b703f4ff19ffc2ebeb3832b91"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "7c2173966c45a478722ad44e28dce24b"
server_key_iv: "01e5cb20ea3ff04d1dae88201e1c6aea"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.591850 9215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:06.611688 9215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:06.613580 9215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:06.613742 9215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:06.618139 9215 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:41545
I20260808 07:15:06.618181 9287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:41545 every 8 connection(s)
I20260808 07:15:06.618551 9215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:06.619428 9288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:06.621716 9288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91: Bootstrap starting.
I20260808 07:15:06.622318 9288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:06.622617 9288 log.cc:824] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:06.623377 9288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91: No bootstrap required, opened a new log
I20260808 07:15:06.625058 9288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d155f5b703f4ff19ffc2ebeb3832b91" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41545 } }
I20260808 07:15:06.625196 9288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:06.625218 9288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d155f5b703f4ff19ffc2ebeb3832b91, State: Initialized, Role: FOLLOWER
I20260808 07:15:06.625402 9288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d155f5b703f4ff19ffc2ebeb3832b91" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41545 } }
I20260808 07:15:06.625468 9288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:06.625490 9288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:06.625562 9288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:06.626096 9288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d155f5b703f4ff19ffc2ebeb3832b91" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41545 } }
I20260808 07:15:06.626209 9288 leader_election.cc:302] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d155f5b703f4ff19ffc2ebeb3832b91; no voters:
I20260808 07:15:06.626402 9288 leader_election.cc:288] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:06.626459 9291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:06.626642 9291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [term 1 LEADER]: Becoming Leader. State: Replica: 8d155f5b703f4ff19ffc2ebeb3832b91, State: Running, Role: LEADER
I20260808 07:15:06.626655 9288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:06.626744 9291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d155f5b703f4ff19ffc2ebeb3832b91" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41545 } }
I20260808 07:15:06.627161 9293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d155f5b703f4ff19ffc2ebeb3832b91. Latest consensus state: current_term: 1 leader_uuid: "8d155f5b703f4ff19ffc2ebeb3832b91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d155f5b703f4ff19ffc2ebeb3832b91" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41545 } } }
I20260808 07:15:06.627285 9292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d155f5b703f4ff19ffc2ebeb3832b91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d155f5b703f4ff19ffc2ebeb3832b91" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 41545 } } }
I20260808 07:15:06.627334 9293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:06.627624 9292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:06.627763 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 9215
I20260808 07:15:06.627895 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:06.628165 22538 external_mini_cluster.cc:1428] Setting key 560b59bc466f8e525800fe6402f6c861
W20260808 07:15:06.628633 9310 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:06.628737 9310 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:06.628860 9303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:06.629026 9303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:06.630607 9303 catalog_manager.cc:1347] Generated new cluster ID: 96445b97202342a2ab8d064f6aa0fd24
I20260808 07:15:06.630659 9303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:06 dist-test-slave-6k22 krb5kdc[9202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:06.638190 9303 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:06.638890 9303 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:06.640062 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:06.628652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.22.2.190:41545)
0808 07:15:06.628811 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:06.628821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:06.629003 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:15:06.629119 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:06.629126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:06.629181 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:06.629318 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:06.630159 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.631581 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.632710 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.633340 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.633443 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:06.636289 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:06.636293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:06.636293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:06.636306 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:06.638377 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.638778 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.638781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.638781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.638897 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.639131 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.639132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.639132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.639394 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:06.639588 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:06.639699 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 07:15:06.639849 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260808 07:15:06.643427 9303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91: Generated new TSK 0
I20260808 07:15:06.643633 9303 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:15:06.675175 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41545
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:06.758466 9318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:06.758683 9318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:06.758725 9318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:06.759974 9318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:06.760031 9318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:06.760075 9318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:15:06.761544 9318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:06.761813 9318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:06.762044 9318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:15:06.767277 9318 server_base.cc:1034] running on GCE node
W20260808 07:15:06.767244 9324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:06.767261 9323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:06.767254 9326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:06.767786 9318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:06.768002 9318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:06.769138 9318 hybrid_clock.cc:648] HybridClock initialized: now 1786173306769128 us; error 29 us; skew 500 ppm
Aug 08 07:15:06 dist-test-slave-6k22 krb5kdc[9202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:06.771756 9318 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:15:06.772217 9318 webserver.cc:460] Webserver started at http://127.22.2.129:40847/ using document root <none> and password file <none>
I20260808 07:15:06.772467 9318 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:06.772535 9318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:06.772686 9318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:06.773716 9318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "1b8bf12a9a19407e94632394a57d13e3"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "d06755914ac675e659ea71f13e49c93b"
server_key_iv: "a793b20bea7e9b65810394dd5954b046"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.774080 9318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "1b8bf12a9a19407e94632394a57d13e3"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "d06755914ac675e659ea71f13e49c93b"
server_key_iv: "a793b20bea7e9b65810394dd5954b046"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.775528 9318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:15:06.776320 9334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:06.776516 9318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:06.776613 9318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "1b8bf12a9a19407e94632394a57d13e3"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "d06755914ac675e659ea71f13e49c93b"
server_key_iv: "a793b20bea7e9b65810394dd5954b046"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.776697 9318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:06.797523 9318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:06.799407 9318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:06.799590 9318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:06.799842 9318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:06.800146 9318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:06.800199 9318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:06.800242 9318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:06.800271 9318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:06.806128 9318 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:40207
I20260808 07:15:06.806182 9468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:40207 every 8 connection(s)
I20260808 07:15:06.806494 9318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:15:06.810642 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 9318
I20260808 07:15:06.810776 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:15:06.810981 22538 external_mini_cluster.cc:1428] Setting key fa4d7fbb60ec5fcc73c05bdb1463e311
Aug 08 07:15:06 dist-test-slave-6k22 krb5kdc[9202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:06.816305 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:06.807160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:41647 (local address 127.22.2.190:41545)
0808 07:15:06.807207 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:06.807209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:06.807420 (+ 211us) server_negotiation.cc:407] Connection header received
0808 07:15:06.808023 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:06.808024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:06.808043 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:06.808102 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:06.809053 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.809578 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.810644 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.810931 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.811015 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:06.813493 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:06.813494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:06.813495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:06.813503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:06.814790 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.815290 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.815291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.815292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.815353 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.815674 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.815675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.815675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.815799 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:06.815855 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:06.816170 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 07:15:06.816239 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:06.816655 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:06.807132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41545 (local address 127.22.2.129:41647)
0808 07:15:06.807278 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:06.807328 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:06.807915 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:06.808170 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:06.808174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:06.808754 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:06.808951 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:06.808956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.809676 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.809677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:06.810553 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:06.810555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.811028 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.811029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:06.811189 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:06.811802 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:06.811814 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:06.813394 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:06.814889 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:06.814896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:06.814898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:06.815212 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:06.815427 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:06.815428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:06.815428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:06.815592 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:06.815865 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:06.815871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:06.816031 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:15:06.816135 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:15:06.816323 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260808 07:15:06.817183 9469 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41545
I20260808 07:15:06.817312 9469 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:06.817507 9469 heartbeater.cc:511] Master 127.22.2.190:41545 requested a full tablet report, sending...
I20260808 07:15:06.818045 9249 ts_manager.cc:194] Registered new tserver with Master: 1b8bf12a9a19407e94632394a57d13e3 (127.22.2.129:40207)
I20260808 07:15:06.818666 9249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:41647
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:15:06.838697 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41545
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:06.924532 9474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:06.924764 9474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:06.924805 9474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:06.925976 9474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:06.926047 9474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:06.926095 9474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:15:06.927839 9474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:06.928129 9474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:06.928388 9474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:06.933750 9480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:06.933728 9479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:06.933843 9482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:06.934078 9474 server_base.cc:1034] running on GCE node
I20260808 07:15:06.934298 9474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:06.934533 9474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:06.935616 9474 hybrid_clock.cc:648] HybridClock initialized: now 1786173306935601 us; error 31 us; skew 500 ppm
Aug 08 07:15:06 dist-test-slave-6k22 krb5kdc[9202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:06.938274 9474 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:15:06.938776 9474 webserver.cc:460] Webserver started at http://127.22.2.130:38227/ using document root <none> and password file <none>
I20260808 07:15:06.939052 9474 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:06.939129 9474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:06.939287 9474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:06.940493 9474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "7e57f49ab0d64903bca0b54ef9efcb6e"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "2dff178f09420e212e9e8637ec94af68"
server_key_iv: "436bb5894b2229eea958e9350dddde55"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.940887 9474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "7e57f49ab0d64903bca0b54ef9efcb6e"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "2dff178f09420e212e9e8637ec94af68"
server_key_iv: "436bb5894b2229eea958e9350dddde55"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.942287 9474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:15:06.943049 9491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:06.943216 9474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:06.943310 9474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "7e57f49ab0d64903bca0b54ef9efcb6e"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-6k22"
server_key: "2dff178f09420e212e9e8637ec94af68"
server_key_iv: "436bb5894b2229eea958e9350dddde55"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.943410 9474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:06.969453 9474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:06.972013 9474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:06.972184 9474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:06.972445 9474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:06.972777 9474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:06.972831 9474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:06.972867 9474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:06.972898 9474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:06.979266 9474 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:35621
I20260808 07:15:06.979313 9618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:35621 every 8 connection(s)
I20260808 07:15:06.979727 9474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:15:06.984135 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 9474
I20260808 07:15:06.984268 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:15:06.984457 22538 external_mini_cluster.cc:1428] Setting key 07d53da52368240b04b4ac1dc6be8542
Aug 08 07:15:06 dist-test-slave-6k22 krb5kdc[9202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:06.989748 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:06.980474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:35263 (local address 127.22.2.190:41545)
0808 07:15:06.980536 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:06.980537 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:06.980736 (+ 199us) server_negotiation.cc:407] Connection header received
0808 07:15:06.981274 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:06.981276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:06.981290 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:06.981341 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:06.982288 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.982866 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.983949 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.984155 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.984240 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:06.986706 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:06.986708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:06.986708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:06.986716 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:06.988221 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.988705 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.988706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.988707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.988758 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.989057 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.989058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.989058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.989157 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:06.989207 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:06.989563 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 07:15:06.989622 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:15:06.990023 9621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:06.980411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41545 (local address 127.22.2.130:35263)
0808 07:15:06.980610 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:06.980648 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:06.981162 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:06.981417 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:06.981421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:06.981984 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:06.982177 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:06.982182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.982959 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.982960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:06.983858 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:06.983862 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.984253 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.984255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:06.984420 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:06.984928 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:06.984937 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:06.986614 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:06.988306 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:06.988314 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:06.988316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:06.988629 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:06.988849 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:06.988850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:06.988850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:06.988993 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:06.989222 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:06.989228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:06.989413 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 07:15:06.989545 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 07:15:06.989724 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":137,"threads_started":1}
I20260808 07:15:06.990550 9619 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41545
I20260808 07:15:06.990669 9619 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:06.990854 9619 heartbeater.cc:511] Master 127.22.2.190:41545 requested a full tablet report, sending...
I20260808 07:15:06.991292 9249 ts_manager.cc:194] Registered new tserver with Master: 7e57f49ab0d64903bca0b54ef9efcb6e (127.22.2.130:35621)
I20260808 07:15:06.991732 9249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:35263
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:15:07.010674 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41545
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:07.093379 9624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:07.093624 9624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:07.093663 9624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:07.094887 9624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:07.094951 9624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:07.094998 9624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:15:07.096565 9624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:07.096834 9624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:07.097066 9624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:07.102159 9631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:07.102164 9630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:07.102351 9624 server_base.cc:1034] running on GCE node
W20260808 07:15:07.102164 9633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:07.102613 9624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:07.102895 9624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:07.104053 9624 hybrid_clock.cc:648] HybridClock initialized: now 1786173307104048 us; error 37 us; skew 500 ppm
Aug 08 07:15:07 dist-test-slave-6k22 krb5kdc[9202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173307, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:07.106587 9624 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:15:07.107030 9624 webserver.cc:460] Webserver started at http://127.22.2.131:45571/ using document root <none> and password file <none>
I20260808 07:15:07.107273 9624 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:07.107342 9624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:07.107492 9624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:07.108655 9624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "4a4b044e24f5422d8ab44a7d8210c439"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "78f20fd61f6971e1ddcc332c0957323c"
server_key_iv: "cb1ecd56947dd0ef62cf285b108132a2"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.109027 9624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "4a4b044e24f5422d8ab44a7d8210c439"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "78f20fd61f6971e1ddcc332c0957323c"
server_key_iv: "cb1ecd56947dd0ef62cf285b108132a2"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.110390 9624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:07.111129 9640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.111310 9624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:07.111409 9624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "4a4b044e24f5422d8ab44a7d8210c439"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "78f20fd61f6971e1ddcc332c0957323c"
server_key_iv: "cb1ecd56947dd0ef62cf285b108132a2"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.111498 9624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:07.141144 9624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:07.143836 9624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:07.143999 9624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:07.144258 9624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:07.144590 9624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:07.144646 9624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.144714 9624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:07.144742 9624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.151001 9624 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:32839
I20260808 07:15:07.151047 9767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:32839 every 8 connection(s)
I20260808 07:15:07.151372 9624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:15:07.155978 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 9624
I20260808 07:15:07.156085 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:15:07.156255 22538 external_mini_cluster.cc:1428] Setting key 52d825fc35435bcbf7e61906237d1816
Aug 08 07:15:07 dist-test-slave-6k22 krb5kdc[9202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173307, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:07.160513 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.152082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:44431 (local address 127.22.2.190:41545)
0808 07:15:07.152127 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.152129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.152349 (+ 220us) server_negotiation.cc:407] Connection header received
0808 07:15:07.152890 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.152892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.152905 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.152969 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:07.153963 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.154485 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.155495 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.155789 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.155866 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.158058 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:07.158059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:07.158060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:07.158066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:07.159307 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:07.159757 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:07.159758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:07.159758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:07.159811 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:07.160021 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:07.160021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:07.160022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:07.160118 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:07.160164 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.160388 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.160451 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:07.160748 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.152038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:41545 (local address 127.22.2.131:44431)
0808 07:15:07.152213 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.152251 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:07.152774 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:07.153042 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:07.153046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:07.153637 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:07.153855 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.153860 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.154587 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.154588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.155398 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.155401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.155887 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.155888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.156003 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.156579 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:07.156587 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:07.157974 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:07.159396 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:07.159402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:07.159404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:07.159694 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:07.159872 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:07.159872 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:07.159873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:07.159973 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:07.160169 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:07.160173 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:07.160284 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 07:15:07.160369 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:15:07.160486 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260808 07:15:07.161247 9768 heartbeater.cc:347] Connected to a master server at 127.22.2.190:41545
I20260808 07:15:07.161365 9768 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:07.161547 9768 heartbeater.cc:511] Master 127.22.2.190:41545 requested a full tablet report, sending...
I20260808 07:15:07.161929 9249 ts_manager.cc:194] Registered new tserver with Master: 4a4b044e24f5422d8ab44a7d8210c439 (127.22.2.131:32839)
I20260808 07:15:07.162302 9249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:44431
I20260808 07:15:07.168058 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:15:07.175298 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.169415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.22.2.190:41545)
0808 07:15:07.169458 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.169460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.169598 (+ 138us) server_negotiation.cc:407] Connection header received
0808 07:15:07.169636 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.169637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.169645 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.169693 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:07.170212 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.170676 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.171365 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.171641 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.171716 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.173146 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:07.173147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:07.173147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:07.173154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:07.174315 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:07.174629 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:07.174630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:07.174630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:07.174681 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:07.174852 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:07.174852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:07.174853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:07.174988 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:07.175063 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.175148 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.175205 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:07.176846 9249 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:15:07.178488 9249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:15:07.185393 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.181039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35621 (local address 127.0.0.1:46866)
0808 07:15:07.181592 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.181602 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:07.181653 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:07.182077 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:07.182078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:07.182080 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:07.182119 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.182121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.183331 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.183332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.184290 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.184293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.185175 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.185176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.185235 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.185246 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:15:07.185292 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:15:07.185338 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260808 07:15:07.185459 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.181182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46866 (local address 127.22.2.130:35621)
0808 07:15:07.181910 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.181914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.181924 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:07.181962 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.181964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.181980 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.182063 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:07.182181 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.183221 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.184385 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.185085 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.185159 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.185219 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.185323 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.185363 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":85,"threads_started":1}
I20260808 07:15:07.185876 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.180995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:32839 (local address 127.0.0.1:59376)
0808 07:15:07.181179 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.181216 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:07.181301 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:07.181777 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:07.181781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:07.181783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:07.181867 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.181870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.183199 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.183199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.184513 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.184516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.185487 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.185666 (+ 179us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.185714 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.185717 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:07.185753 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 07:15:07.185782 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":82,"thread_start_us":64,"threads_started":1}
I20260808 07:15:07.186084 9786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.181389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.22.2.131:32839)
0808 07:15:07.181567 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.181572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.181583 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:15:07.181627 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.181629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.181646 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.181761 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:07.181933 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.183094 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.184604 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.185392 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.185789 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.185858 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.185885 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.186004 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260808 07:15:07.186239 9545 tablet_service.cc:1467] Processing CreateTablet for tablet bc453ff54a024ebfa37ba596edafaae0 (DEFAULT_TABLE table=test-table [id=8ac34097ec9a415a9f850781acb3a1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:07.186609 9545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc453ff54a024ebfa37ba596edafaae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:07.186743 9694 tablet_service.cc:1467] Processing CreateTablet for tablet bc453ff54a024ebfa37ba596edafaae0 (DEFAULT_TABLE table=test-table [id=8ac34097ec9a415a9f850781acb3a1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:07.187098 9694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc453ff54a024ebfa37ba596edafaae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:07.187362 9785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.181250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:40207 (local address 127.0.0.1:35540)
0808 07:15:07.181701 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.181713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:07.181767 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:07.182106 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:07.182107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:07.182108 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:07.182137 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.182138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.183073 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.183074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.184526 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.184528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.187064 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.187065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.187131 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.187135 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:07.187253 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 07:15:07.187297 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":75,"thread_start_us":71,"threads_started":1}
I20260808 07:15:07.187408 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.181389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.22.2.129:40207)
0808 07:15:07.181584 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.181588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.181945 (+ 357us) server_negotiation.cc:407] Connection header received
0808 07:15:07.181985 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.181987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.182000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.182211 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:07.182283 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.182972 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.186209 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.186970 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.187040 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.187093 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.187279 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.187330 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":114,"threads_started":1}
I20260808 07:15:07.188344 9389 tablet_service.cc:1467] Processing CreateTablet for tablet bc453ff54a024ebfa37ba596edafaae0 (DEFAULT_TABLE table=test-table [id=8ac34097ec9a415a9f850781acb3a1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:07.188681 9389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc453ff54a024ebfa37ba596edafaae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:07.189719 9790 tablet_bootstrap.cc:492] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Bootstrap starting.
I20260808 07:15:07.190362 9790 tablet_bootstrap.cc:654] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:07.190559 9791 tablet_bootstrap.cc:492] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Bootstrap starting.
I20260808 07:15:07.191009 9790 log.cc:824] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:07.191255 9791 tablet_bootstrap.cc:654] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:07.191819 9791 log.cc:824] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:07.191890 9790 tablet_bootstrap.cc:492] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: No bootstrap required, opened a new log
I20260808 07:15:07.191954 9790 ts_tablet_manager.cc:1397] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:15:07.192181 9793 tablet_bootstrap.cc:492] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3: Bootstrap starting.
I20260808 07:15:07.192739 9793 tablet_bootstrap.cc:654] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:07.193220 9793 log.cc:824] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:07.193442 9790 raft_consensus.cc:348] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.193605 9790 raft_consensus.cc:374] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:07.193651 9790 raft_consensus.cc:729] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e57f49ab0d64903bca0b54ef9efcb6e, State: Initialized, Role: FOLLOWER
I20260808 07:15:07.193763 9790 consensus_queue.cc:260] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.194002 9793 tablet_bootstrap.cc:492] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3: No bootstrap required, opened a new log
I20260808 07:15:07.194082 9793 ts_tablet_manager.cc:1397] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:15:07.194090 9619 heartbeater.cc:503] Master 127.22.2.190:41545 was elected leader, sending a full tablet report...
I20260808 07:15:07.194088 9790 ts_tablet_manager.cc:1428] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:15:07.195926 9793 raft_consensus.cc:348] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.196089 9793 raft_consensus.cc:374] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:07.196136 9793 raft_consensus.cc:729] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8bf12a9a19407e94632394a57d13e3, State: Initialized, Role: FOLLOWER
I20260808 07:15:07.196252 9793 consensus_queue.cc:260] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.196530 9793 ts_tablet_manager.cc:1428] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:15:07.196570 9469 heartbeater.cc:503] Master 127.22.2.190:41545 was elected leader, sending a full tablet report...
I20260808 07:15:07.196971 9791 tablet_bootstrap.cc:492] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: No bootstrap required, opened a new log
I20260808 07:15:07.197053 9791 ts_tablet_manager.cc:1397] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 07:15:07.198500 9791 raft_consensus.cc:348] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.198652 9791 raft_consensus.cc:374] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:07.198698 9791 raft_consensus.cc:729] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a4b044e24f5422d8ab44a7d8210c439, State: Initialized, Role: FOLLOWER
I20260808 07:15:07.198807 9791 consensus_queue.cc:260] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.199076 9791 ts_tablet_manager.cc:1428] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:15:07.199095 9768 heartbeater.cc:503] Master 127.22.2.190:41545 was elected leader, sending a full tablet report...
W20260808 07:15:07.230278 9620 tablet.cc:2369] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:15:07.307258 9470 tablet.cc:2369] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:15:07.354822 9799 raft_consensus.cc:482] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:15:07.355005 9799 raft_consensus.cc:504] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.355417 9799 leader_election.cc:288] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e57f49ab0d64903bca0b54ef9efcb6e (127.22.2.130:35621), 1b8bf12a9a19407e94632394a57d13e3 (127.22.2.129:40207)
I20260808 07:15:07.358533 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.355884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:40781 (local address 127.22.2.129:40207)
0808 07:15:07.355937 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.355939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.356063 (+ 124us) server_negotiation.cc:407] Connection header received
0808 07:15:07.356115 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.356116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.356129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.356186 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:07.356351 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.356826 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.357776 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.358138 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.358194 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.358201 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.358396 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.358445 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:15:07.358536 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.355764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:40207 (local address 127.22.2.131:40781)
0808 07:15:07.355977 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.355989 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:07.356055 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:07.356201 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:07.356202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:07.356203 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:07.356272 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.356273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.356907 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.356908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.357676 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.357678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.358222 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.358224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.358300 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.358305 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:15:07.358383 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:15:07.358418 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":126,"threads_started":1}
I20260808 07:15:07.358608 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.355732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:45879 (local address 127.22.2.130:35621)
0808 07:15:07.355804 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.355807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.355820 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:15:07.355874 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.355875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.355906 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.355987 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:07.356201 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.356741 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.357784 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.358209 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.358282 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.358291 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.358491 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.358533 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:15:07.358613 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.355636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:35621 (local address 127.22.2.131:45879)
0808 07:15:07.355704 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.355717 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:07.355767 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:07.355993 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:07.355994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:07.355996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:07.356094 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.356096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.356830 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.356832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.357673 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.357675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.358293 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.358294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.358404 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.358407 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:07.358463 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:15:07.358501 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 07:15:07.358929 9567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc453ff54a024ebfa37ba596edafaae0" candidate_uuid: "4a4b044e24f5422d8ab44a7d8210c439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" is_pre_election: true
I20260808 07:15:07.358911 9418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc453ff54a024ebfa37ba596edafaae0" candidate_uuid: "4a4b044e24f5422d8ab44a7d8210c439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8bf12a9a19407e94632394a57d13e3" is_pre_election: true
I20260808 07:15:07.359082 9567 raft_consensus.cc:2454] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a4b044e24f5422d8ab44a7d8210c439 in term 0.
I20260808 07:15:07.359082 9418 raft_consensus.cc:2454] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a4b044e24f5422d8ab44a7d8210c439 in term 0.
I20260808 07:15:07.359313 9642 leader_election.cc:302] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a4b044e24f5422d8ab44a7d8210c439, 7e57f49ab0d64903bca0b54ef9efcb6e; no voters:
I20260808 07:15:07.359462 9799 raft_consensus.cc:2781] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:15:07.359551 9799 raft_consensus.cc:482] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:15:07.359614 9799 raft_consensus.cc:3037] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:07.360363 9799 raft_consensus.cc:504] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.360503 9799 leader_election.cc:288] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [CANDIDATE]: Term 1 election: Requested vote from peers 7e57f49ab0d64903bca0b54ef9efcb6e (127.22.2.130:35621), 1b8bf12a9a19407e94632394a57d13e3 (127.22.2.129:40207)
I20260808 07:15:07.360750 9418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc453ff54a024ebfa37ba596edafaae0" candidate_uuid: "4a4b044e24f5422d8ab44a7d8210c439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8bf12a9a19407e94632394a57d13e3"
I20260808 07:15:07.360736 9567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc453ff54a024ebfa37ba596edafaae0" candidate_uuid: "4a4b044e24f5422d8ab44a7d8210c439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e"
I20260808 07:15:07.360838 9418 raft_consensus.cc:3037] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:07.360848 9567 raft_consensus.cc:3037] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:07.361786 9418 raft_consensus.cc:2454] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a4b044e24f5422d8ab44a7d8210c439 in term 1.
I20260808 07:15:07.361786 9567 raft_consensus.cc:2454] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a4b044e24f5422d8ab44a7d8210c439 in term 1.
I20260808 07:15:07.361960 9642 leader_election.cc:302] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b8bf12a9a19407e94632394a57d13e3, 4a4b044e24f5422d8ab44a7d8210c439; no voters:
I20260808 07:15:07.362067 9799 raft_consensus.cc:2781] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:07.362218 9799 raft_consensus.cc:686] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 1 LEADER]: Becoming Leader. State: Replica: 4a4b044e24f5422d8ab44a7d8210c439, State: Running, Role: LEADER
I20260808 07:15:07.362321 9799 consensus_queue.cc:237] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } }
I20260808 07:15:07.363191 9249 catalog_manager.cc:5495] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a4b044e24f5422d8ab44a7d8210c439 (127.22.2.131). New cstate: current_term: 1 leader_uuid: "4a4b044e24f5422d8ab44a7d8210c439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4b044e24f5422d8ab44a7d8210c439" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 32839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 } health_report { overall_health: UNKNOWN } } }
W20260808 07:15:07.401988 9769 tablet.cc:2369] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:15:07.410643 9786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.408388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.22.2.131:32839)
0808 07:15:07.408451 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.408453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.408462 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:15:07.408529 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.408530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.408550 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.408610 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:07.408760 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.409226 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.409942 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.410122 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.410172 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.410303 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:07.410396 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:07.410454 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.410492 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.410510 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:15:07.412467 9567 raft_consensus.cc:1261] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 1 FOLLOWER]: Refusing update from remote peer 4a4b044e24f5422d8ab44a7d8210c439: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:07.412528 9418 raft_consensus.cc:1261] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 1 FOLLOWER]: Refusing update from remote peer 4a4b044e24f5422d8ab44a7d8210c439: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:07.412905 9801 consensus_queue.cc:1035] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e57f49ab0d64903bca0b54ef9efcb6e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:07.413058 9799 consensus_queue.cc:1035] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b8bf12a9a19407e94632394a57d13e3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 40207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:07.415617 9805 mvcc.cc:205] Tried to move back new op lower bound from 7316165867158818816 to 7316165866956836864. Current Snapshot: MvccSnapshot[applied={T|T < 7316165867158818816}]
I20260808 07:15:07.417531 9249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:07.417631 9249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:07.418459 9249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d155f5b703f4ff19ffc2ebeb3832b91: Sending DeleteTablet for 3 replicas of tablet bc453ff54a024ebfa37ba596edafaae0
I20260808 07:15:07.418913 9389 tablet_service.cc:1514] Processing DeleteTablet for tablet bc453ff54a024ebfa37ba596edafaae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:35540
I20260808 07:15:07.418941 9694 tablet_service.cc:1514] Processing DeleteTablet for tablet bc453ff54a024ebfa37ba596edafaae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:59376
I20260808 07:15:07.418964 9545 tablet_service.cc:1514] Processing DeleteTablet for tablet bc453ff54a024ebfa37ba596edafaae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:07 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:46866
I20260808 07:15:07.419178 9817 tablet_replica.cc:330] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3: stopping tablet replica
I20260808 07:15:07.419212 9816 tablet_replica.cc:330] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: stopping tablet replica
I20260808 07:15:07.419274 9817 raft_consensus.cc:2229] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:07.419297 9816 raft_consensus.cc:2229] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:15:07.419396 9816 raft_consensus.cc:2258] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:07.419392 9817 raft_consensus.cc:2258] T bc453ff54a024ebfa37ba596edafaae0 P 1b8bf12a9a19407e94632394a57d13e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:07.419474 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 9318
I20260808 07:15:07.421121 9818 tablet_replica.cc:330] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: stopping tablet replica
I20260808 07:15:07.421248 9818 raft_consensus.cc:2229] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:07.421403 9818 raft_consensus.cc:2258] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:07.421691 9816 ts_tablet_manager.cc:1905] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:07.422346 9818 ts_tablet_manager.cc:1905] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:07.423841 9816 ts_tablet_manager.cc:1918] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:07.423911 9816 log.cc:1196] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/bc453ff54a024ebfa37ba596edafaae0
I20260808 07:15:07.424067 9818 ts_tablet_manager.cc:1918] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:07.424130 9818 log.cc:1196] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/bc453ff54a024ebfa37ba596edafaae0
I20260808 07:15:07.424202 9816 ts_tablet_manager.cc:1939] T bc453ff54a024ebfa37ba596edafaae0 P 4a4b044e24f5422d8ab44a7d8210c439: Deleting consensus metadata
I20260808 07:15:07.424396 9818 ts_tablet_manager.cc:1939] T bc453ff54a024ebfa37ba596edafaae0 P 7e57f49ab0d64903bca0b54ef9efcb6e: Deleting consensus metadata
I20260808 07:15:07.424619 9234 catalog_manager.cc:4841] TS 4a4b044e24f5422d8ab44a7d8210c439 (127.22.2.131:32839): tablet bc453ff54a024ebfa37ba596edafaae0 (table test-table [id=8ac34097ec9a415a9f850781acb3a1d7]) successfully deleted
I20260808 07:15:07.424795 9233 catalog_manager.cc:4841] TS 7e57f49ab0d64903bca0b54ef9efcb6e (127.22.2.130:35621): tablet bc453ff54a024ebfa37ba596edafaae0 (table test-table [id=8ac34097ec9a415a9f850781acb3a1d7]) successfully deleted
W20260808 07:15:07.430212 9233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:40207 (error 108)
I20260808 07:15:07.430493 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 9474
I20260808 07:15:07.432528 9785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.432390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:40207 (local address 127.0.0.1:35542)
0808 07:15:07.432443 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.432481 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:40207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 07:15:07.432644 9233 catalog_manager.cc:4584] TS 1b8bf12a9a19407e94632394a57d13e3 (127.22.2.129:40207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc453ff54a024ebfa37ba596edafaae0: Network error: Client connection negotiation failed: client connection to 127.22.2.129:40207: connect: Connection refused (error 111)
I20260808 07:15:07.436394 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 9624
I20260808 07:15:07.443205 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 9215
2026-08-08T07:15:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2207 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T07:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:07Z Disabled control of system clock
I20260808 07:15:07.473791 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35985
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:15:07.555069 9824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:07.555291 9824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:07.555311 9824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:07.556453 9824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:07.556490 9824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:07.556504 9824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:07.556516 9824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:07.556528 9824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:07.557788 9824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35985
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:07.557981 9824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:07.558218 9824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:07.563025 9834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:07.563056 9831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:07.563268 9824 server_base.cc:1034] running on GCE node
W20260808 07:15:07.563073 9830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:07.563627 9824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:07.563943 9824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:07.565106 9824 hybrid_clock.cc:648] HybridClock initialized: now 1786173307565103 us; error 34 us; skew 500 ppm
I20260808 07:15:07.566373 9824 webserver.cc:460] Webserver started at http://127.22.2.190:38467/ using document root <none> and password file <none>
I20260808 07:15:07.566627 9824 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:07.566700 9824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:07.566859 9824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:07.568085 9824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "d84f753845554f84b64e21e66347f138"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "2ab99dee4a36e1250bd81d3bb2c71b68"
server_key_iv: "4a406dc275d6d75e673cf0c514faf8cf"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.568452 9824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "d84f753845554f84b64e21e66347f138"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "2ab99dee4a36e1250bd81d3bb2c71b68"
server_key_iv: "4a406dc275d6d75e673cf0c514faf8cf"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.569747 9824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:07.570482 9840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.570652 9824 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:07.570757 9824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "d84f753845554f84b64e21e66347f138"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "2ab99dee4a36e1250bd81d3bb2c71b68"
server_key_iv: "4a406dc275d6d75e673cf0c514faf8cf"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.570853 9824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:07.594317 9824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:07.594689 9824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:07.594867 9824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:07.599118 9824 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35985
I20260808 07:15:07.599155 9901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35985 every 8 connection(s)
I20260808 07:15:07.599525 9824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:07.600502 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:07.602751 9902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138: Bootstrap starting.
I20260808 07:15:07.603339 9902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:07.603718 9902 log.cc:824] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:07.604408 9902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138: No bootstrap required, opened a new log
I20260808 07:15:07.606004 9902 raft_consensus.cc:348] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f753845554f84b64e21e66347f138" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35985 } }
I20260808 07:15:07.606139 9902 raft_consensus.cc:374] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:07.606161 9902 raft_consensus.cc:729] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84f753845554f84b64e21e66347f138, State: Initialized, Role: FOLLOWER
I20260808 07:15:07.606331 9902 consensus_queue.cc:260] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f753845554f84b64e21e66347f138" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35985 } }
I20260808 07:15:07.606391 9902 raft_consensus.cc:388] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:07.606444 9902 raft_consensus.cc:482] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:07.606493 9902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:07.607045 9902 raft_consensus.cc:504] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f753845554f84b64e21e66347f138" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35985 } }
I20260808 07:15:07.607152 9902 leader_election.cc:302] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d84f753845554f84b64e21e66347f138; no voters:
I20260808 07:15:07.607342 9902 leader_election.cc:288] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:07.607439 9907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:07.607630 9902 sys_catalog.cc:564] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:07.607636 9907 raft_consensus.cc:686] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [term 1 LEADER]: Becoming Leader. State: Replica: d84f753845554f84b64e21e66347f138, State: Running, Role: LEADER
I20260808 07:15:07.607816 9907 consensus_queue.cc:237] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f753845554f84b64e21e66347f138" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35985 } }
I20260808 07:15:07.608330 9908 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d84f753845554f84b64e21e66347f138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f753845554f84b64e21e66347f138" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35985 } } }
I20260808 07:15:07.608371 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 9824
I20260808 07:15:07.608425 9908 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:07.608475 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:07.608690 22538 external_mini_cluster.cc:1428] Setting key 0093b7c4601ccb0f21f2371198ed3142
I20260808 07:15:07.608770 9912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:07.608961 9909 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d84f753845554f84b64e21e66347f138. Latest consensus state: current_term: 1 leader_uuid: "d84f753845554f84b64e21e66347f138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84f753845554f84b64e21e66347f138" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35985 } } }
I20260808 07:15:07.609131 9909 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:07.609283 9912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:07.612052 9912 catalog_manager.cc:1347] Generated new cluster ID: 7ae0d72b3b8b40a191b6376bd1ce8dfb
I20260808 07:15:07.612112 9912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:07.614957 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.609451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.22.2.190:35985)
0808 07:15:07.609661 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.609671 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.609710 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:15:07.610037 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.610046 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.610128 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.610556 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:07.610772 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.611662 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.612761 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.613387 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.613533 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.614236 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:07.614240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:07.614240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:07.614451 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:07.614497 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:07.614512 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:07.614585 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.614653 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.614732 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260808 07:15:07.622331 9912 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:07.622995 9912 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:07.628351 9912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138: Generated new TSK 0
I20260808 07:15:07.628515 9912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:07.632922 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35985
--builtin_ntp_servers=127.22.2.148:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:15:07.716128 9935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:07.716369 9935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:07.716406 9935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:07.717542 9935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:07.717604 9935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:07.717649 9935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:15:07.719038 9935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:07.719307 9935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:07.719664 9935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:07.725682 9940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:07.725835 9935 server_base.cc:1034] running on GCE node
W20260808 07:15:07.725771 9941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:07.725958 9943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:07.726179 9935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:07.726444 9935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:07.727658 9935 hybrid_clock.cc:648] HybridClock initialized: now 1786173307727618 us; error 48 us; skew 500 ppm
I20260808 07:15:07.735406 9935 webserver.cc:460] Webserver started at http://127.22.2.129:40105/ using document root <none> and password file <none>
I20260808 07:15:07.735756 9935 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:07.735848 9935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:07.736002 9935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:07.737167 9935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "74ee429ce52b290269962728248f1a8b"
server_key_iv: "e0c7fd3a0890b330bf2d7ec422cd51bb"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.737560 9935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "74ee429ce52b290269962728248f1a8b"
server_key_iv: "e0c7fd3a0890b330bf2d7ec422cd51bb"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.738950 9935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:07.739790 9949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.739984 9935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:07.740088 9935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "74ee429ce52b290269962728248f1a8b"
server_key_iv: "e0c7fd3a0890b330bf2d7ec422cd51bb"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.740177 9935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:07.755632 9935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:07.755944 9935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:07.756093 9935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:07.756322 9935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:07.756668 9935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:07.756731 9935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.756775 9935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:07.756808 9935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.762846 9935 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:33171
I20260808 07:15:07.762887 10070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:33171 every 8 connection(s)
I20260808 07:15:07.763206 9935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:15:07.768239 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.763907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:47219 (local address 127.22.2.190:35985)
0808 07:15:07.763998 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.764000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.764248 (+ 248us) server_negotiation.cc:407] Connection header received
0808 07:15:07.764769 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.764770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.764783 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.764835 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:07.765516 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.766011 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.766943 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.767127 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.767180 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.767997 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:07.767998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:07.767998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:07.768002 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:07.768014 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:07.768019 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:07.768063 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.768147 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.768171 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:15:07.768409 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.763869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35985 (local address 127.22.2.129:47219)
0808 07:15:07.764087 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.764146 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:07.764682 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:07.764904 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:07.764908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:07.764916 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:07.765442 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.765446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.766090 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.766091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.766868 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.766870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.767201 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.767202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.767312 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.767862 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:07.767862 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:07.767890 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:07.767912 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:07.768074 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:07.768078 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:07.768081 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:07.768133 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:15:07.768213 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260808 07:15:07.768800 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 9935
I20260808 07:15:07.768898 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:15:07.768962 10071 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35985
I20260808 07:15:07.769052 10071 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:07.769079 22538 external_mini_cluster.cc:1428] Setting key 5ec468b6cf01032843bc0d020ea530a1
I20260808 07:15:07.769193 10071 heartbeater.cc:511] Master 127.22.2.190:35985 requested a full tablet report, sending...
I20260808 07:15:07.769723 9857 ts_manager.cc:194] Registered new tserver with Master: 9ce2d39b016f4a48a145f7de6a7cdbb3 (127.22.2.129:33171)
I20260808 07:15:07.770032 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35985
--builtin_ntp_servers=127.22.2.148:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:15:07.770648 9857 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.129:47219
W20260808 07:15:07.861368 10074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:07.861655 10074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:07.861689 10074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:07.863620 10074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:07.863672 10074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:07.863729 10074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:15:07.865855 10074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:07.866109 10074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:07.866425 10074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:07.872113 10083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:07.872141 10084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:07.872063 10086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:07.872394 10074 server_base.cc:1034] running on GCE node
I20260808 07:15:07.872637 10074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:07.872916 10074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:07.874064 10074 hybrid_clock.cc:648] HybridClock initialized: now 1786173307874050 us; error 52 us; skew 500 ppm
I20260808 07:15:07.875675 10074 webserver.cc:460] Webserver started at http://127.22.2.130:37469/ using document root <none> and password file <none>
I20260808 07:15:07.875945 10074 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:07.876008 10074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:07.876144 10074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:07.877604 10074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "2793076967c244c781fbabf97cf078c3"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "1981634bb065a67018f6dd8754756910"
server_key_iv: "bd208d36ef479d0887d9139ec60df608"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.878022 10074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "2793076967c244c781fbabf97cf078c3"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "1981634bb065a67018f6dd8754756910"
server_key_iv: "bd208d36ef479d0887d9139ec60df608"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.879849 10074 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 07:15:07.880784 10092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.880995 10074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:07.881091 10074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "2793076967c244c781fbabf97cf078c3"
format_stamp: "Formatted at 2026-08-08 07:15:07 on dist-test-slave-6k22"
server_key: "1981634bb065a67018f6dd8754756910"
server_key_iv: "bd208d36ef479d0887d9139ec60df608"
server_key_version: "encryptionkey@0"
I20260808 07:15:07.881165 10074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:07.893728 10074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:07.894058 10074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:07.894256 10074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:07.894551 10074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:07.894960 10074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:07.895042 10074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.895084 10074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:07.895123 10074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:07.902309 10074 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:45159
I20260808 07:15:07.902360 10222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:45159 every 8 connection(s)
I20260808 07:15:07.902724 10074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:15:07.905918 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 10074
I20260808 07:15:07.906046 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:15:07.906271 22538 external_mini_cluster.cc:1428] Setting key 33ab49619a4f8c5a32dcf7ad7e5f433a
I20260808 07:15:07.907635 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35985
--builtin_ntp_servers=127.22.2.148:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:15:07.908321 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.903483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53251 (local address 127.22.2.190:35985)
0808 07:15:07.903549 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:07.903551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:07.903837 (+ 286us) server_negotiation.cc:407] Connection header received
0808 07:15:07.904305 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:07.904306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:07.904318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:07.904382 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:07.905003 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.905546 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.906640 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.906966 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.907040 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.907995 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:07.907996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:07.907997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:07.908003 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:07.908020 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:07.908026 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:07.908089 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:07.908158 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:15:07.908188 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:15:07.908419 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:07.903480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35985 (local address 127.22.2.130:53251)
0808 07:15:07.903713 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:07.903745 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:07.904214 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:07.904435 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:07.904438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:07.904447 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:07.904922 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.904926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.905631 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.905632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.906548 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:07.906551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:07.907050 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:07.907051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:07.907200 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:07.907848 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:07.907849 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:07.907888 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:07.907918 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:07.908096 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:07.908100 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:07.908103 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:07.908155 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:15:07.908234 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":137,"threads_started":1}
I20260808 07:15:07.908895 10223 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35985
I20260808 07:15:07.909008 10223 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:07.909184 10223 heartbeater.cc:511] Master 127.22.2.190:35985 requested a full tablet report, sending...
I20260808 07:15:07.909667 9857 ts_manager.cc:194] Registered new tserver with Master: 2793076967c244c781fbabf97cf078c3 (127.22.2.130:45159)
I20260808 07:15:07.910079 9857 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.130:53251
W20260808 07:15:07.997232 10226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:07.997437 10226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:07.997459 10226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:07.998605 10226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:07.998651 10226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:07.998675 10226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:15:08.000075 10226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:08.000320 10226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:08.000608 10226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:08.005615 10235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:08.005656 10232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:08.005705 10231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:08.006043 10226 server_base.cc:1034] running on GCE node
I20260808 07:15:08.006335 10226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:08.006577 10226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:08.007738 10226 hybrid_clock.cc:648] HybridClock initialized: now 1786173308007787 us; error 91 us; skew 500 ppm
I20260808 07:15:08.009164 10226 webserver.cc:460] Webserver started at http://127.22.2.131:42417/ using document root <none> and password file <none>
I20260808 07:15:08.009439 10226 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:08.009503 10226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:08.009637 10226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:08.010716 10226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "9c2bbd8338654b4ea1549134122f58d8"
format_stamp: "Formatted at 2026-08-08 07:15:08 on dist-test-slave-6k22"
server_key: "5fb9c749e4b374f5d3d1f09f8d096e38"
server_key_iv: "ab786902e3125caf69a35131333f2232"
server_key_version: "encryptionkey@0"
I20260808 07:15:08.011086 10226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "9c2bbd8338654b4ea1549134122f58d8"
format_stamp: "Formatted at 2026-08-08 07:15:08 on dist-test-slave-6k22"
server_key: "5fb9c749e4b374f5d3d1f09f8d096e38"
server_key_iv: "ab786902e3125caf69a35131333f2232"
server_key_version: "encryptionkey@0"
I20260808 07:15:08.012466 10226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:08.013193 10242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:08.013365 10226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:08.013460 10226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "9c2bbd8338654b4ea1549134122f58d8"
format_stamp: "Formatted at 2026-08-08 07:15:08 on dist-test-slave-6k22"
server_key: "5fb9c749e4b374f5d3d1f09f8d096e38"
server_key_iv: "ab786902e3125caf69a35131333f2232"
server_key_version: "encryptionkey@0"
I20260808 07:15:08.013542 10226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:08.022073 10226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:08.022344 10226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:08.022487 10226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:08.022699 10226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:08.023012 10226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:08.023064 10226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:08.023108 10226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:08.023133 10226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:08.028946 10226 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:33217
I20260808 07:15:08.028996 10368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:33217 every 8 connection(s)
I20260808 07:15:08.029311 10226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:15:08.034228 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 10226
I20260808 07:15:08.034361 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:15:08.034601 22538 external_mini_cluster.cc:1428] Setting key 7593ed63ce995edff9fbdab5a7234412
I20260808 07:15:08.034823 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.030003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:47097 (local address 127.22.2.190:35985)
0808 07:15:08.030050 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.030052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.030308 (+ 256us) server_negotiation.cc:407] Connection header received
0808 07:15:08.030855 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.030856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.030869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.030928 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:08.031638 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.032156 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.033339 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.033539 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.033626 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.034529 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:08.034530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:08.034531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:08.034536 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:08.034556 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:08.034560 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:08.034622 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.034715 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.034751 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:08.035089 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.029978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35985 (local address 127.22.2.131:47097)
0808 07:15:08.030190 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:08.030223 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:08.030739 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:08.031000 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:08.031004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:08.031012 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:08.031505 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.031509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.032262 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.032263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.033233 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.033236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.033637 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.033639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.033768 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.034348 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:08.034349 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:08.034388 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:08.034436 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:08.034630 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:08.034635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:08.034638 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:08.034714 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:15:08.034848 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260808 07:15:08.035503 10369 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35985
I20260808 07:15:08.035662 10369 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:08.035872 10369 heartbeater.cc:511] Master 127.22.2.190:35985 requested a full tablet report, sending...
I20260808 07:15:08.036255 9857 ts_manager.cc:194] Registered new tserver with Master: 9c2bbd8338654b4ea1549134122f58d8 (127.22.2.131:33217)
I20260808 07:15:08.036618 9857 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.131:47097
I20260808 07:15:08.046430 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:15:08.050814 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.047811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43136 (local address 127.22.2.190:35985)
0808 07:15:08.047865 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.047867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.048038 (+ 171us) server_negotiation.cc:407] Connection header received
0808 07:15:08.048106 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.048107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.048120 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.048192 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:08.048377 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.048872 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.049627 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.049817 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.049902 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.050549 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:08.050550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:08.050550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:08.050555 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:08.050568 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:08.050572 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:08.050640 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.050699 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.050724 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:15:08.052299 9857 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:15:08.053776 9857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:15:08.060017 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.055889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:45159 (local address 127.0.0.1:41588)
0808 07:15:08.056226 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:08.056268 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:08.056347 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:08.056655 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:08.056661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:08.056664 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:08.056731 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.056733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.057602 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.057603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.058719 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.058722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.059708 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.059709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.059786 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.059800 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:15:08.059887 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:15:08.059938 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":73,"thread_start_us":65,"threads_started":1}
I20260808 07:15:08.060051 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.055986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41588 (local address 127.22.2.130:45159)
0808 07:15:08.056135 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.056141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.056357 (+ 216us) server_negotiation.cc:407] Connection header received
0808 07:15:08.056483 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.056486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.056501 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.056580 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:08.056794 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.057510 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.058820 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.059619 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.059696 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.059790 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.059915 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.059959 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260808 07:15:08.060284 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.055999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.22.2.131:33217)
0808 07:15:08.056230 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.056234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.056412 (+ 178us) server_negotiation.cc:407] Connection header received
0808 07:15:08.056462 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.056464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.056482 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.056565 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:08.056829 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.057510 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.059025 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.059761 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.059832 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.059925 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.060134 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.060180 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":87,"threads_started":1}
I20260808 07:15:08.060503 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.055892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:33217 (local address 127.0.0.1:58992)
0808 07:15:08.056226 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:08.056268 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:08.056347 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:08.056653 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:08.056657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:08.056659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:08.056746 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.056748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.057608 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.057609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.058934 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.058937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.059961 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.059962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.060031 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.060036 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:15:08.060390 (+ 354us) client_negotiation.cc:240] Negotiation successful
0808 07:15:08.060435 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":32,"thread_start_us":98,"threads_started":1}
I20260808 07:15:08.060756 10151 tablet_service.cc:1467] Processing CreateTablet for tablet 693190efef7b45d1b136e9ed5d0b4a10 (DEFAULT_TABLE table=test-table [id=37d221f596c14b18b3a76a3d53e3a59a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:08.061168 10151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 693190efef7b45d1b136e9ed5d0b4a10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:08.061160 10297 tablet_service.cc:1467] Processing CreateTablet for tablet 693190efef7b45d1b136e9ed5d0b4a10 (DEFAULT_TABLE table=test-table [id=37d221f596c14b18b3a76a3d53e3a59a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:08.061529 10297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 693190efef7b45d1b136e9ed5d0b4a10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:08.064751 10390 tablet_bootstrap.cc:492] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: Bootstrap starting.
I20260808 07:15:08.064971 10392 tablet_bootstrap.cc:492] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: Bootstrap starting.
I20260808 07:15:08.065438 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.056101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33171 (local address 127.0.0.1:43516)
0808 07:15:08.056432 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:08.056443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:08.056513 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:08.057858 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:08.057859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:08.057861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:08.057903 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.057905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.059122 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.059123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.060012 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.060015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.065202 (+ 5187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.065204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.065264 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.065268 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:08.065325 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:15:08.065366 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260808 07:15:08.065467 10389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.056233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.22.2.129:33171)
0808 07:15:08.057690 (+ 1457us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.057694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.057704 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:08.057745 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.057747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.057762 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.057853 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:08.057974 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.059036 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.060109 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.065129 (+ 5020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.065202 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.065288 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.065347 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.065385 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1413,"thread_start_us":83,"threads_started":1}
I20260808 07:15:08.065558 10390 tablet_bootstrap.cc:654] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:08.065641 10392 tablet_bootstrap.cc:654] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:08.065915 10390 log.cc:824] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:08.065960 10392 log.cc:824] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:08.066392 9999 tablet_service.cc:1467] Processing CreateTablet for tablet 693190efef7b45d1b136e9ed5d0b4a10 (DEFAULT_TABLE table=test-table [id=37d221f596c14b18b3a76a3d53e3a59a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:08.066725 10392 tablet_bootstrap.cc:492] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: No bootstrap required, opened a new log
I20260808 07:15:08.066749 9999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 693190efef7b45d1b136e9ed5d0b4a10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:08.066795 10392 ts_tablet_manager.cc:1397] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:15:08.068290 10390 tablet_bootstrap.cc:492] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: No bootstrap required, opened a new log
I20260808 07:15:08.068382 10390 ts_tablet_manager.cc:1397] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:15:08.069437 10392 raft_consensus.cc:348] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.069614 10392 raft_consensus.cc:374] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:08.069653 10392 raft_consensus.cc:729] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2bbd8338654b4ea1549134122f58d8, State: Initialized, Role: FOLLOWER
I20260808 07:15:08.069794 10392 consensus_queue.cc:260] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.070079 10392 ts_tablet_manager.cc:1428] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260808 07:15:08.070199 10369 heartbeater.cc:503] Master 127.22.2.190:35985 was elected leader, sending a full tablet report...
I20260808 07:15:08.070292 10396 tablet_bootstrap.cc:492] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3: Bootstrap starting.
I20260808 07:15:08.070514 10390 raft_consensus.cc:348] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.070694 10390 raft_consensus.cc:374] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:08.070745 10390 raft_consensus.cc:729] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2793076967c244c781fbabf97cf078c3, State: Initialized, Role: FOLLOWER
I20260808 07:15:08.070883 10390 consensus_queue.cc:260] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.071013 10396 tablet_bootstrap.cc:654] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:08.071173 10390 ts_tablet_manager.cc:1428] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:15:08.071230 10223 heartbeater.cc:503] Master 127.22.2.190:35985 was elected leader, sending a full tablet report...
I20260808 07:15:08.071506 10396 log.cc:824] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:08.072314 10396 tablet_bootstrap.cc:492] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3: No bootstrap required, opened a new log
I20260808 07:15:08.072367 10396 ts_tablet_manager.cc:1397] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:15:08.074019 10396 raft_consensus.cc:348] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.074131 10396 raft_consensus.cc:374] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:08.074151 10396 raft_consensus.cc:729] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce2d39b016f4a48a145f7de6a7cdbb3, State: Initialized, Role: FOLLOWER
I20260808 07:15:08.074259 10396 consensus_queue.cc:260] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.074469 10396 ts_tablet_manager.cc:1428] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:15:08.074556 10071 heartbeater.cc:503] Master 127.22.2.190:35985 was elected leader, sending a full tablet report...
I20260808 07:15:08.079099 10397 raft_consensus.cc:482] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:15:08.079169 10397 raft_consensus.cc:504] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.079427 10397 leader_election.cc:288] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2bbd8338654b4ea1549134122f58d8 (127.22.2.131:33217), 9ce2d39b016f4a48a145f7de6a7cdbb3 (127.22.2.129:33171)
I20260808 07:15:08.083109 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.079551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:33217 (local address 127.22.2.130:41205)
0808 07:15:08.079623 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:08.079635 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:08.079678 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:08.079900 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:08.079901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:08.079903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:08.079985 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.079986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.080866 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.080867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.082136 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.082139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.082862 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.082864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.082935 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.082939 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:08.082989 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:15:08.083038 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":40}
I20260808 07:15:08.083125 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.079697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:41205 (local address 127.22.2.131:33217)
0808 07:15:08.079754 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.079756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.079763 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:08.079790 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.079791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.079800 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.079874 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:08.080054 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.080775 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.082227 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.082768 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.082851 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.082858 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.083008 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.083047 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:15:08.083177 10389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.079624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:44621 (local address 127.22.2.129:33171)
0808 07:15:08.079664 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.079666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.079779 (+ 113us) server_negotiation.cc:407] Connection header received
0808 07:15:08.079841 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.079842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.079854 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.079903 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:08.080051 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.080775 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.082227 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.082768 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.082847 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.082855 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.083068 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.083109 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:08.083386 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.079594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33171 (local address 127.22.2.130:44621)
0808 07:15:08.079729 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:08.079738 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:08.079783 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:08.079913 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:08.079914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:08.079915 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:08.079989 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.079990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.080866 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.080867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.082141 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:08.082143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.082907 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.082908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:08.082982 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.082985 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:08.083267 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 07:15:08.083318 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":56,"thread_start_us":80,"threads_started":1}
I20260808 07:15:08.083412 10321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "693190efef7b45d1b136e9ed5d0b4a10" candidate_uuid: "2793076967c244c781fbabf97cf078c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2bbd8338654b4ea1549134122f58d8" is_pre_election: true
I20260808 07:15:08.083590 10321 raft_consensus.cc:2454] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2793076967c244c781fbabf97cf078c3 in term 0.
I20260808 07:15:08.083665 10021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "693190efef7b45d1b136e9ed5d0b4a10" candidate_uuid: "2793076967c244c781fbabf97cf078c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" is_pre_election: true
I20260808 07:15:08.083797 10021 raft_consensus.cc:2454] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2793076967c244c781fbabf97cf078c3 in term 0.
I20260808 07:15:08.083843 10096 leader_election.cc:302] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2793076967c244c781fbabf97cf078c3, 9c2bbd8338654b4ea1549134122f58d8; no voters:
I20260808 07:15:08.084040 10397 raft_consensus.cc:2781] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:15:08.084085 10397 raft_consensus.cc:482] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:15:08.084100 10397 raft_consensus.cc:3037] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:08.084832 10397 raft_consensus.cc:504] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.085090 10397 leader_election.cc:288] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2bbd8338654b4ea1549134122f58d8 (127.22.2.131:33217), 9ce2d39b016f4a48a145f7de6a7cdbb3 (127.22.2.129:33171)
I20260808 07:15:08.085109 10321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "693190efef7b45d1b136e9ed5d0b4a10" candidate_uuid: "2793076967c244c781fbabf97cf078c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2bbd8338654b4ea1549134122f58d8"
I20260808 07:15:08.085143 10021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "693190efef7b45d1b136e9ed5d0b4a10" candidate_uuid: "2793076967c244c781fbabf97cf078c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3"
I20260808 07:15:08.085183 10321 raft_consensus.cc:3037] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:08.085235 10021 raft_consensus.cc:3037] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:08.086091 10321 raft_consensus.cc:2454] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2793076967c244c781fbabf97cf078c3 in term 1.
I20260808 07:15:08.086091 10021 raft_consensus.cc:2454] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2793076967c244c781fbabf97cf078c3 in term 1.
I20260808 07:15:08.086289 10094 leader_election.cc:302] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2793076967c244c781fbabf97cf078c3, 9ce2d39b016f4a48a145f7de6a7cdbb3; no voters:
I20260808 07:15:08.086395 10397 raft_consensus.cc:2781] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:08.086529 10397 raft_consensus.cc:686] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 1 LEADER]: Becoming Leader. State: Replica: 2793076967c244c781fbabf97cf078c3, State: Running, Role: LEADER
I20260808 07:15:08.086622 10397 consensus_queue.cc:237] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } }
I20260808 07:15:08.087306 9857 catalog_manager.cc:5495] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2793076967c244c781fbabf97cf078c3 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "2793076967c244c781fbabf97cf078c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2793076967c244c781fbabf97cf078c3" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 45159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260808 07:15:08.094373 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.092340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41592 (local address 127.22.2.130:45159)
0808 07:15:08.092378 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.092380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.092389 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:15:08.092449 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.092450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.092464 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.092512 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:08.092607 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.093044 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.093717 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.093895 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.093943 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.094071 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:08.094153 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:08.094209 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.094248 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.094261 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:08.096022 10021 raft_consensus.cc:1261] T 693190efef7b45d1b136e9ed5d0b4a10 P 9ce2d39b016f4a48a145f7de6a7cdbb3 [term 1 FOLLOWER]: Refusing update from remote peer 2793076967c244c781fbabf97cf078c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:08.096051 10321 raft_consensus.cc:1261] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 1 FOLLOWER]: Refusing update from remote peer 2793076967c244c781fbabf97cf078c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:08.096334 10401 consensus_queue.cc:1035] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ce2d39b016f4a48a145f7de6a7cdbb3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:08.096446 10397 consensus_queue.cc:1035] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2bbd8338654b4ea1549134122f58d8" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 33217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:08.101286 9859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:08.101397 9859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:08.102162 9859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d84f753845554f84b64e21e66347f138: Sending DeleteTablet for 3 replicas of tablet 693190efef7b45d1b136e9ed5d0b4a10
I20260808 07:15:08.102555 10297 tablet_service.cc:1514] Processing DeleteTablet for tablet 693190efef7b45d1b136e9ed5d0b4a10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:08 UTC) from {username='slave'} at 127.0.0.1:58992
I20260808 07:15:08.102620 10151 tablet_service.cc:1514] Processing DeleteTablet for tablet 693190efef7b45d1b136e9ed5d0b4a10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:08 UTC) from {username='slave'} at 127.0.0.1:41588
I20260808 07:15:08.102816 10416 tablet_replica.cc:330] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: stopping tablet replica
I20260808 07:15:08.102841 9999 tablet_service.cc:1514] Processing DeleteTablet for tablet 693190efef7b45d1b136e9ed5d0b4a10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:08 UTC) from {username='slave'} at 127.0.0.1:43516
I20260808 07:15:08.102993 10416 raft_consensus.cc:2229] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:15:08.103093 10416 raft_consensus.cc:2258] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:08.103101 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 9935
I20260808 07:15:08.104743 10416 ts_tablet_manager.cc:1905] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:08.107254 10416 ts_tablet_manager.cc:1918] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:08.107285 10415 tablet_replica.cc:330] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: stopping tablet replica
I20260808 07:15:08.107393 10415 raft_consensus.cc:2229] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:08.107409 10416 log.cc:1196] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/693190efef7b45d1b136e9ed5d0b4a10
I20260808 07:15:08.107508 10415 raft_consensus.cc:2258] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:08.107858 10416 ts_tablet_manager.cc:1939] T 693190efef7b45d1b136e9ed5d0b4a10 P 2793076967c244c781fbabf97cf078c3: Deleting consensus metadata
I20260808 07:15:08.108049 10415 ts_tablet_manager.cc:1905] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:08.108390 9841 catalog_manager.cc:4841] TS 2793076967c244c781fbabf97cf078c3 (127.22.2.130:45159): tablet 693190efef7b45d1b136e9ed5d0b4a10 (table test-table [id=37d221f596c14b18b3a76a3d53e3a59a]) successfully deleted
I20260808 07:15:08.109714 10415 ts_tablet_manager.cc:1918] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:08.109788 10415 log.cc:1196] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/693190efef7b45d1b136e9ed5d0b4a10
I20260808 07:15:08.110057 10415 ts_tablet_manager.cc:1939] T 693190efef7b45d1b136e9ed5d0b4a10 P 9c2bbd8338654b4ea1549134122f58d8: Deleting consensus metadata
I20260808 07:15:08.111291 9843 catalog_manager.cc:4841] TS 9c2bbd8338654b4ea1549134122f58d8 (127.22.2.131:33217): tablet 693190efef7b45d1b136e9ed5d0b4a10 (table test-table [id=37d221f596c14b18b3a76a3d53e3a59a]) successfully deleted
W20260808 07:15:08.112219 9841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:33171 (error 108)
I20260808 07:15:08.112480 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 10074
I20260808 07:15:08.113371 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.112564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:33171 (local address 127.0.0.1:43524)
0808 07:15:08.113281 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:08.113332 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":695}
W20260808 07:15:08.114460 9841 catalog_manager.cc:4584] TS 9ce2d39b016f4a48a145f7de6a7cdbb3 (127.22.2.129:33171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 693190efef7b45d1b136e9ed5d0b4a10: Network error: Client connection negotiation failed: client connection to 127.22.2.129:33171: connect: Connection refused (error 111)
I20260808 07:15:08.124464 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 10226
I20260808 07:15:08.132094 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 9824
2026-08-08T07:15:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (688 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:08 dist-test-slave-6k22 krb5kdc[10421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:08 dist-test-slave-6k22 krb5kdc[10421](info): set up 2 sockets
Aug 08 07:15:08 dist-test-slave-6k22 krb5kdc[10421](info): commencing operation
krb5kdc: starting...
W20260808 07:15:09.253823 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:09.380384 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35845
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:35845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:09.463912 10434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:09.464195 10434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:09.464242 10434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:09.465643 10434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:09.465749 10434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:09.465782 10434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:09.465819 10434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:09.465850 10434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:09.467435 10434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:35845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:35845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:09.467754 10434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:09.467998 10434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:09.473438 10439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:09.473501 10434 server_base.cc:1034] running on GCE node
W20260808 07:15:09.473429 10440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:09.473436 10442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:09.473872 10434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:09.474128 10434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:09.475288 10434 hybrid_clock.cc:648] HybridClock initialized: now 1786173309475275 us; error 36 us; skew 500 ppm
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:09.477744 10434 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:09.478159 10434 webserver.cc:460] Webserver started at http://127.22.2.190:46505/ using document root <none> and password file <none>
I20260808 07:15:09.478392 10434 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:09.478458 10434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:09.478626 10434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:09.479768 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "b8f8250e5d724b32986474dbf430befa"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "a43e8a3a5a0b500e1f7c9770e95eab00"
server_key_iv: "f3fddb8e1baf4b12337fe3fc423a39e3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.480135 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "b8f8250e5d724b32986474dbf430befa"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "a43e8a3a5a0b500e1f7c9770e95eab00"
server_key_iv: "f3fddb8e1baf4b12337fe3fc423a39e3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.481463 10434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:09.482156 10449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.482321 10434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:09.482430 10434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "b8f8250e5d724b32986474dbf430befa"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "a43e8a3a5a0b500e1f7c9770e95eab00"
server_key_iv: "f3fddb8e1baf4b12337fe3fc423a39e3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.482506 10434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:09.489249 10434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:09.490958 10434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:09.491107 10434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:09.495201 10434 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:35845
I20260808 07:15:09.495234 10509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:35845 every 8 connection(s)
I20260808 07:15:09.495647 10434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:09.496491 10511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:09.498718 10511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa: Bootstrap starting.
I20260808 07:15:09.499280 10511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:09.499598 10511 log.cc:824] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:09.500275 10511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa: No bootstrap required, opened a new log
I20260808 07:15:09.501878 10511 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f8250e5d724b32986474dbf430befa" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35845 } }
I20260808 07:15:09.502007 10511 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:09.502058 10511 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f8250e5d724b32986474dbf430befa, State: Initialized, Role: FOLLOWER
I20260808 07:15:09.502214 10511 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f8250e5d724b32986474dbf430befa" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35845 } }
I20260808 07:15:09.502292 10511 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:09.502341 10511 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:09.502393 10511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:09.502888 10511 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f8250e5d724b32986474dbf430befa" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35845 } }
I20260808 07:15:09.502991 10511 leader_election.cc:302] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8f8250e5d724b32986474dbf430befa; no voters:
I20260808 07:15:09.503162 10511 leader_election.cc:288] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:09.503211 10515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:09.503361 10515 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [term 1 LEADER]: Becoming Leader. State: Replica: b8f8250e5d724b32986474dbf430befa, State: Running, Role: LEADER
I20260808 07:15:09.503405 10511 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:09.503446 10515 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f8250e5d724b32986474dbf430befa" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35845 } }
I20260808 07:15:09.503809 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8f8250e5d724b32986474dbf430befa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f8250e5d724b32986474dbf430befa" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35845 } } }
I20260808 07:15:09.503907 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:09.503939 10517 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8f8250e5d724b32986474dbf430befa. Latest consensus state: current_term: 1 leader_uuid: "b8f8250e5d724b32986474dbf430befa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f8250e5d724b32986474dbf430befa" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 35845 } } }
I20260808 07:15:09.504002 10517 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:09.504140 10523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:09.504693 10523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:09.505651 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 10434
I20260808 07:15:09.505765 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:09.505973 22538 external_mini_cluster.cc:1428] Setting key 8e14a01070217a243556bd5ac374812a
I20260808 07:15:09.506698 10523 catalog_manager.cc:1347] Generated new cluster ID: 5b5e133a4aae452c9426649bcb8d0bf1
I20260808 07:15:09.506762 10523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:09.515075 10523 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:09.515791 10523 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:09.517007 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.506448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36490 (local address 127.22.2.190:35845)
0808 07:15:09.507740 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:09.507747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:09.507782 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:15:09.507890 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:09.507895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:09.507943 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:09.508076 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:09.508703 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.509627 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.510406 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.510821 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.510929 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:09.513369 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:09.513373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:09.513373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:09.513385 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:09.515275 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:09.515655 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:09.515658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:09.515659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:09.515754 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:09.516026 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:09.516027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:09.516027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:09.516322 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:09.516500 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:09.516577 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:15:09.516834 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1260,"thread_start_us":76,"threads_started":1}
I20260808 07:15:09.518923 10523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa: Generated new TSK 0
I20260808 07:15:09.519081 10523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:15:09.555531 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35845
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:09.653403 10542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:09.653636 10542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:09.653677 10542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:09.654920 10542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:09.654964 10542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:09.654991 10542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:15:09.656504 10542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:09.656777 10542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:09.656994 10542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:09.662690 10548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:09.662710 10549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:09.662705 10551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:09.662953 10542 server_base.cc:1034] running on GCE node
I20260808 07:15:09.663137 10542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:09.663380 10542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:09.664537 10542 hybrid_clock.cc:648] HybridClock initialized: now 1786173309664513 us; error 55 us; skew 500 ppm
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:09.666945 10542 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:15:09.667367 10542 webserver.cc:460] Webserver started at http://127.22.2.129:45703/ using document root <none> and password file <none>
I20260808 07:15:09.667645 10542 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:09.667714 10542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:09.667873 10542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:09.668921 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "b0a2049231364843ba944b91f1a725d3"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "da881fc1b523d7f02aaeb6b1d00d3ad6"
server_key_iv: "c555e5b78bf5e0d19a3922366704f3ce"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.669304 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "b0a2049231364843ba944b91f1a725d3"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "da881fc1b523d7f02aaeb6b1d00d3ad6"
server_key_iv: "c555e5b78bf5e0d19a3922366704f3ce"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.670607 10542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:09.671322 10558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.671505 10542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:09.671633 10542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "b0a2049231364843ba944b91f1a725d3"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "da881fc1b523d7f02aaeb6b1d00d3ad6"
server_key_iv: "c555e5b78bf5e0d19a3922366704f3ce"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.671712 10542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:09.680228 10542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:09.682008 10542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:09.682157 10542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:09.682372 10542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:09.682667 10542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:09.682714 10542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.682753 10542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:09.682783 10542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.689796 10542 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:37765
I20260808 07:15:09.689877 10688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:37765 every 8 connection(s)
I20260808 07:15:09.690225 10542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:15:09.691901 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 10542
I20260808 07:15:09.692025 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:15:09.692241 22538 external_mini_cluster.cc:1428] Setting key f0a235eb9f09fdda00849c9bfa2710fc
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:09.700505 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.690991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:56535 (local address 127.22.2.190:35845)
0808 07:15:09.691054 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:09.691056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:09.691268 (+ 212us) server_negotiation.cc:407] Connection header received
0808 07:15:09.691845 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:09.691846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:09.691863 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:09.691919 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:09.693040 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.693790 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.694778 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.695120 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.695205 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:09.697694 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:09.697695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:09.697696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:09.697703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:09.699088 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:09.699499 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:09.699501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:09.699501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:09.699599 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:09.699850 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:09.699851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:09.699852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:09.699964 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:09.700037 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:09.700306 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 07:15:09.700413 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:15:09.700675 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.690969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35845 (local address 127.22.2.129:56535)
0808 07:15:09.691126 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:09.691179 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:09.691713 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:09.691985 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:09.691989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:09.692655 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:09.692913 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:09.692919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.693888 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.693889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:09.694684 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:09.694686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.695214 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.695215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:09.695401 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:09.695971 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:09.695982 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:09.697595 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:09.699187 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:09.699195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:09.699198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:09.699425 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:09.699655 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:09.699656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:09.699656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:09.699774 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:09.700029 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:09.700036 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:09.700177 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 07:15:09.700283 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:15:09.700438 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260808 07:15:09.701231 10689 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35845
I20260808 07:15:09.701336 10689 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
I20260808 07:15:09.701537 10689 heartbeater.cc:511] Master 127.22.2.190:35845 requested a full tablet report, sending...
I20260808 07:15:09.702061 10468 ts_manager.cc:194] Registered new tserver with Master: b0a2049231364843ba944b91f1a725d3 (127.22.2.129:37765)
I20260808 07:15:09.702723 10468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:56535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:15:09.718979 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35845
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:09.801558 10695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:09.801786 10695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:09.801826 10695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:09.803005 10695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:09.803066 10695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:09.803109 10695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:15:09.804752 10695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:09.805014 10695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:09.805248 10695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:09.810469 10701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:09.810484 10702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:09.810484 10704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:09.810599 10695 server_base.cc:1034] running on GCE node
I20260808 07:15:09.810953 10695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:09.811224 10695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:09.812392 10695 hybrid_clock.cc:648] HybridClock initialized: now 1786173309812378 us; error 40 us; skew 500 ppm
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:09.815071 10695 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:15:09.815538 10695 webserver.cc:460] Webserver started at http://127.22.2.130:42961/ using document root <none> and password file <none>
I20260808 07:15:09.815836 10695 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:09.815904 10695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:09.816073 10695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:09.817260 10695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "bf34b885de6d430ab5664b2b01b9f47e"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "4bdaf7024ee443cf45e3ebda13bf7137"
server_key_iv: "54ea8011886e34a2cec29ad77fdf83b0"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.817641 10695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "bf34b885de6d430ab5664b2b01b9f47e"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "4bdaf7024ee443cf45e3ebda13bf7137"
server_key_iv: "54ea8011886e34a2cec29ad77fdf83b0"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.819020 10695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:09.819958 10711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.820128 10695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:15:09.820235 10695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "bf34b885de6d430ab5664b2b01b9f47e"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "4bdaf7024ee443cf45e3ebda13bf7137"
server_key_iv: "54ea8011886e34a2cec29ad77fdf83b0"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.820307 10695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:09.834416 10695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:09.836261 10695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:09.836421 10695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:09.836645 10695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:09.836951 10695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:09.837000 10695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.837041 10695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:09.837069 10695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.842816 10695 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:44317
I20260808 07:15:09.842885 10836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:44317 every 8 connection(s)
I20260808 07:15:09.843194 10695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:15:09.844131 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 10695
I20260808 07:15:09.844249 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:15:09.844448 22538 external_mini_cluster.cc:1428] Setting key 61f0dd2864ce69e56fc9c1f039955b1d
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:09.852478 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.843973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:46055 (local address 127.22.2.190:35845)
0808 07:15:09.844046 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:09.844049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:09.844204 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:15:09.844740 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:09.844741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:09.844754 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:09.844804 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:09.845739 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.846235 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.847248 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.847447 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.847523 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:09.849625 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:09.849626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:09.849626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:09.849634 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:09.851061 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:09.851452 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:09.851453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:09.851453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:09.851512 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:09.851777 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:09.851778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:09.851778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:09.851905 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:09.852011 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:09.852293 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 07:15:09.852378 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:15:09.852710 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.843922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35845 (local address 127.22.2.130:46055)
0808 07:15:09.844086 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:09.844127 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:09.844633 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:09.844886 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:09.844890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:09.845452 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:09.845640 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:09.845645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.846344 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.846345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:09.847155 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:09.847158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.847532 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.847533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:09.847686 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:09.848182 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:09.848190 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:09.849531 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:09.851142 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:09.851149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:09.851152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:09.851377 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:09.851591 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:09.851592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:09.851592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:09.851709 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:09.852012 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:09.852020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:09.852164 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 07:15:09.852270 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:15:09.852443 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
I20260808 07:15:09.853207 10837 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35845
I20260808 07:15:09.853313 10837 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:09.853499 10837 heartbeater.cc:511] Master 127.22.2.190:35845 requested a full tablet report, sending...
I20260808 07:15:09.853888 10468 ts_manager.cc:194] Registered new tserver with Master: bf34b885de6d430ab5664b2b01b9f47e (127.22.2.130:44317)
I20260808 07:15:09.854317 10468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:46055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:15:09.870944 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35845
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:37419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:09.956324 10842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:09.956533 10842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:09.956558 10842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:09.957702 10842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:09.957739 10842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:09.957764 10842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:15:09.959169 10842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:09.959359 10842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:09.959538 10842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:09.964514 10847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:09.964484 10851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:09.964496 10848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:09.964813 10842 server_base.cc:1034] running on GCE node
I20260808 07:15:09.965046 10842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:09.965279 10842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:09.966437 10842 hybrid_clock.cc:648] HybridClock initialized: now 1786173309966421 us; error 27 us; skew 500 ppm
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:09.968968 10842 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:15:09.969403 10842 webserver.cc:460] Webserver started at http://127.22.2.131:43309/ using document root <none> and password file <none>
I20260808 07:15:09.969657 10842 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:09.969725 10842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:09.969877 10842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:09.970965 10842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "6d19695e22e04abc940316b5aa57271d"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "aa0f0838f17578488a4a55007976ec80"
server_key_iv: "f7bfeee029ee16e390efc54fca735af3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.971324 10842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "6d19695e22e04abc940316b5aa57271d"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "aa0f0838f17578488a4a55007976ec80"
server_key_iv: "f7bfeee029ee16e390efc54fca735af3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.972781 10842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260808 07:15:09.973536 10858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.973703 10842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:09.973802 10842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "6d19695e22e04abc940316b5aa57271d"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-6k22"
server_key: "aa0f0838f17578488a4a55007976ec80"
server_key_iv: "f7bfeee029ee16e390efc54fca735af3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.973892 10842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:09.983415 10842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:09.985507 10842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:09.985764 10842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:09.986021 10842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:09.986352 10842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:09.986407 10842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.986457 10842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:09.986500 10842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.993094 10842 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:41487
I20260808 07:15:09.993163 10982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:41487 every 8 connection(s)
I20260808 07:15:09.993532 10842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:15:09.996430 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 10842
I20260808 07:15:09.996562 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:15:09.996765 22538 external_mini_cluster.cc:1428] Setting key 80252212db5f5262a0607f2a535cc6aa
Aug 08 07:15:09 dist-test-slave-6k22 krb5kdc[10421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:10.002839 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.994236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:41921 (local address 127.22.2.190:35845)
0808 07:15:09.994320 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:09.994323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:09.994533 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:15:09.995042 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:09.995043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:09.995054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:09.995103 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:09.996167 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.996759 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.998079 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.998263 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.998324 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.000513 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:10.000514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:10.000515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:10.000521 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:10.001720 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:10.002075 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:10.002076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:10.002076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:10.002127 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:10.002327 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:10.002328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:10.002328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:10.002438 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:10.002489 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.002710 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.002773 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:15:10.003111 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.994234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:35845 (local address 127.22.2.131:41921)
0808 07:15:09.994411 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:09.994449 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:09.994955 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:09.995171 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:09.995174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:09.995874 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:09.996080 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:09.996085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.996859 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.996861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:09.997999 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:09.998002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:09.998340 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:09.998341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:09.998458 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:09.999001 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:09.999010 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:10.000422 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:10.001791 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:10.001797 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:10.001799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:10.002014 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:10.002181 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:10.002181 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:10.002182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:10.002276 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:10.002491 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:10.002495 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:10.002605 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.002685 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.002801 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
I20260808 07:15:10.003624 10983 heartbeater.cc:347] Connected to a master server at 127.22.2.190:35845
I20260808 07:15:10.003731 10983 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:10.003914 10983 heartbeater.cc:511] Master 127.22.2.190:35845 requested a full tablet report, sending...
I20260808 07:15:10.004292 10468 ts_manager.cc:194] Registered new tserver with Master: 6d19695e22e04abc940316b5aa57271d (127.22.2.131:41487)
I20260808 07:15:10.004683 10468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:41921
I20260808 07:15:10.008607 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:15:10.016181 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.009931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36492 (local address 127.22.2.190:35845)
0808 07:15:10.009989 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.009991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.010123 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:15:10.010198 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.010198 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.010210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.010273 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:10.010827 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.011293 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.012135 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.012321 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.012420 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.013722 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:10.013724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:10.013724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:10.013732 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:10.014879 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:10.015282 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:10.015283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:10.015284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:10.015359 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:10.015625 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:10.015626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:10.015627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:10.015766 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:10.015884 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.016014 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.016079 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:15:10.017673 10468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:15:10.019395 10468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:15:10.026098 10998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.021869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:41487 (local address 127.0.0.1:60640)
0808 07:15:10.022037 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.022044 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.022090 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.022476 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.022477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.022478 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:10.022511 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.022512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.023922 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.023923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.025165 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.025167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.025790 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.025791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.025869 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.025882 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.025952 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.026004 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":98,"thread_start_us":75,"threads_started":1}
I20260808 07:15:10.026224 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.022141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.22.2.129:37765)
0808 07:15:10.022366 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.022371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.022384 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:15:10.022430 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.022433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.022450 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.022752 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:10.022829 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.023680 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.025061 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.025815 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.025943 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.026014 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.026074 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.026123 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260808 07:15:10.026284 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.022091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60640 (local address 127.22.2.131:41487)
0808 07:15:10.022313 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.022317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.022327 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:10.022365 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.022367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.022382 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.022584 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:10.022655 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.023786 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.025232 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.025706 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.025792 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.025862 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.026143 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.026183 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":127,"threads_started":1}
I20260808 07:15:10.026271 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.021594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44317 (local address 127.0.0.1:57594)
0808 07:15:10.021826 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.021865 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.021934 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.022392 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.022396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.022399 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:10.022483 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.022485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.023931 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.023932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.025092 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.025095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.026010 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.026011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.026082 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.026086 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.026135 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.026177 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
I20260808 07:15:10.026185 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.021607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37765 (local address 127.0.0.1:49646)
0808 07:15:10.021869 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.021878 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.021934 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.022569 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.022570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.022573 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:10.022648 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.022651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.023785 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.023786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.024967 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.024970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.025946 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.025947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.026000 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.026002 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.026062 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.026093 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":51,"thread_start_us":100,"threads_started":1}
I20260808 07:15:10.026599 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.021778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.22.2.130:44317)
0808 07:15:10.022183 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.022188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.022195 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:10.022248 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.022250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.022268 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.022566 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:10.022653 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.023814 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.025176 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.025905 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.026422 (+ 517us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.026475 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.026498 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.026535 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":97,"threads_started":1}
I20260808 07:15:10.027385 10607 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5d2071443c4944a70dccb2f1c838f4 (DEFAULT_TABLE table=test-table [id=83e12fb165574a04871f9b20ca529ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:10.027420 10762 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5d2071443c4944a70dccb2f1c838f4 (DEFAULT_TABLE table=test-table [id=83e12fb165574a04871f9b20ca529ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:10.027446 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5d2071443c4944a70dccb2f1c838f4 (DEFAULT_TABLE table=test-table [id=83e12fb165574a04871f9b20ca529ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:10.027791 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c5d2071443c4944a70dccb2f1c838f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:10.027791 10607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c5d2071443c4944a70dccb2f1c838f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:10.027935 10762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c5d2071443c4944a70dccb2f1c838f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:10.031343 11001 tablet_bootstrap.cc:492] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3: Bootstrap starting.
I20260808 07:15:10.031607 11002 tablet_bootstrap.cc:492] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Bootstrap starting.
I20260808 07:15:10.031638 11003 tablet_bootstrap.cc:492] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Bootstrap starting.
I20260808 07:15:10.032220 11003 tablet_bootstrap.cc:654] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:10.032235 11002 tablet_bootstrap.cc:654] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:10.032697 11003 log.cc:824] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:10.032732 11002 log.cc:824] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:10.033450 11003 tablet_bootstrap.cc:492] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: No bootstrap required, opened a new log
I20260808 07:15:10.033530 11002 tablet_bootstrap.cc:492] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: No bootstrap required, opened a new log
I20260808 07:15:10.033529 11003 ts_tablet_manager.cc:1397] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:15:10.033563 11001 tablet_bootstrap.cc:654] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:10.033643 11002 ts_tablet_manager.cc:1397] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:15:10.034327 11001 log.cc:824] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:10.035027 11001 tablet_bootstrap.cc:492] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3: No bootstrap required, opened a new log
I20260808 07:15:10.035118 11001 ts_tablet_manager.cc:1397] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 07:15:10.035507 11003 raft_consensus.cc:348] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
I20260808 07:15:10.035596 11002 raft_consensus.cc:348] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
I20260808 07:15:10.035732 11003 raft_consensus.cc:374] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:10.035743 11002 raft_consensus.cc:374] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:10.035776 11002 raft_consensus.cc:729] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf34b885de6d430ab5664b2b01b9f47e, State: Initialized, Role: FOLLOWER
I20260808 07:15:10.035776 11003 raft_consensus.cc:729] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d19695e22e04abc940316b5aa57271d, State: Initialized, Role: FOLLOWER
I20260808 07:15:10.035925 11003 consensus_queue.cc:260] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
I20260808 07:15:10.035949 11002 consensus_queue.cc:260] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
I20260808 07:15:10.036218 11002 ts_tablet_manager.cc:1428] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:15:10.036219 11003 ts_tablet_manager.cc:1428] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:15:10.036340 10983 heartbeater.cc:503] Master 127.22.2.190:35845 was elected leader, sending a full tablet report...
I20260808 07:15:10.036378 10837 heartbeater.cc:503] Master 127.22.2.190:35845 was elected leader, sending a full tablet report...
I20260808 07:15:10.037254 11001 raft_consensus.cc:348] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
I20260808 07:15:10.037451 11001 raft_consensus.cc:374] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:10.037603 11001 raft_consensus.cc:729] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a2049231364843ba944b91f1a725d3, State: Initialized, Role: FOLLOWER
I20260808 07:15:10.037753 11001 consensus_queue.cc:260] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
I20260808 07:15:10.038007 11001 ts_tablet_manager.cc:1428] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:15:10.038039 10689 heartbeater.cc:503] Master 127.22.2.190:35845 was elected leader, sending a full tablet report...
W20260808 07:15:10.093776 10838 tablet.cc:2369] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:15:10.190999 10690 tablet.cc:2369] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:15:10.235747 11007 raft_consensus.cc:482] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:15:10.235877 11007 raft_consensus.cc:504] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
I20260808 07:15:10.236191 11007 leader_election.cc:288] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0a2049231364843ba944b91f1a725d3 (127.22.2.129:37765), 6d19695e22e04abc940316b5aa57271d (127.22.2.131:41487)
I20260808 07:15:10.240062 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.236527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:50299 (local address 127.22.2.129:37765)
0808 07:15:10.236577 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.236579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.236591 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:15:10.236623 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.236623 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.236635 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.236692 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:10.236875 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.237635 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.239108 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.239694 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.239772 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.239781 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.239950 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.239989 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:15:10.240062 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.236523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:47373 (local address 127.22.2.131:41487)
0808 07:15:10.236592 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.236593 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.236772 (+ 179us) server_negotiation.cc:407] Connection header received
0808 07:15:10.236815 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.236816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.236823 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.236889 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:10.237000 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.237678 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.239108 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.239695 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.239773 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.239781 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.239952 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.239989 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:15:10.240075 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.236413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37765 (local address 127.22.2.130:50299)
0808 07:15:10.236465 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.236478 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.236547 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.236717 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.236718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.236720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:10.236803 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.236805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.237732 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.237733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.239014 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.239017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.239775 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.239776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.239865 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.239869 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.239929 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.239976 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":9}
I20260808 07:15:10.240082 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.236413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:41487 (local address 127.22.2.130:47373)
0808 07:15:10.236648 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.236664 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.236734 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.236900 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.236901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.236902 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:10.236954 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.236955 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.237756 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.237757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.239014 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.239017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.239775 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.239776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.239865 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.239869 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.239929 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.239976 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":115,"thread_start_us":102,"threads_started":1}
I20260808 07:15:10.240487 10631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c5d2071443c4944a70dccb2f1c838f4" candidate_uuid: "bf34b885de6d430ab5664b2b01b9f47e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a2049231364843ba944b91f1a725d3" is_pre_election: true
I20260808 07:15:10.240533 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c5d2071443c4944a70dccb2f1c838f4" candidate_uuid: "bf34b885de6d430ab5664b2b01b9f47e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d19695e22e04abc940316b5aa57271d" is_pre_election: true
I20260808 07:15:10.240665 10631 raft_consensus.cc:2454] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf34b885de6d430ab5664b2b01b9f47e in term 0.
I20260808 07:15:10.240665 10931 raft_consensus.cc:2454] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf34b885de6d430ab5664b2b01b9f47e in term 0.
I20260808 07:15:10.240890 10713 leader_election.cc:302] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d19695e22e04abc940316b5aa57271d, bf34b885de6d430ab5664b2b01b9f47e; no voters:
I20260808 07:15:10.241073 11007 raft_consensus.cc:2781] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:15:10.241113 11007 raft_consensus.cc:482] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:15:10.241128 11007 raft_consensus.cc:3037] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:10.241832 11007 raft_consensus.cc:504] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
I20260808 07:15:10.241971 11007 leader_election.cc:288] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [CANDIDATE]: Term 1 election: Requested vote from peers b0a2049231364843ba944b91f1a725d3 (127.22.2.129:37765), 6d19695e22e04abc940316b5aa57271d (127.22.2.131:41487)
I20260808 07:15:10.242115 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c5d2071443c4944a70dccb2f1c838f4" candidate_uuid: "bf34b885de6d430ab5664b2b01b9f47e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d19695e22e04abc940316b5aa57271d"
I20260808 07:15:10.242161 10631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c5d2071443c4944a70dccb2f1c838f4" candidate_uuid: "bf34b885de6d430ab5664b2b01b9f47e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a2049231364843ba944b91f1a725d3"
I20260808 07:15:10.242197 10931 raft_consensus.cc:3037] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:10.242249 10631 raft_consensus.cc:3037] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:10.242946 10931 raft_consensus.cc:2454] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf34b885de6d430ab5664b2b01b9f47e in term 1.
I20260808 07:15:10.242951 10631 raft_consensus.cc:2454] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf34b885de6d430ab5664b2b01b9f47e in term 1.
I20260808 07:15:10.243114 10713 leader_election.cc:302] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d19695e22e04abc940316b5aa57271d, bf34b885de6d430ab5664b2b01b9f47e; no voters:
I20260808 07:15:10.243213 11007 raft_consensus.cc:2781] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:10.243376 11007 raft_consensus.cc:686] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 1 LEADER]: Becoming Leader. State: Replica: bf34b885de6d430ab5664b2b01b9f47e, State: Running, Role: LEADER
I20260808 07:15:10.243474 11007 consensus_queue.cc:237] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } }
W20260808 07:15:10.244150 10984 tablet.cc:2369] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:15:10.244371 10467 catalog_manager.cc:5495] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e reported cstate change: term changed from 0 to 1, leader changed from <none> to bf34b885de6d430ab5664b2b01b9f47e (127.22.2.130). New cstate: current_term: 1 leader_uuid: "bf34b885de6d430ab5664b2b01b9f47e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf34b885de6d430ab5664b2b01b9f47e" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 } health_report { overall_health: UNKNOWN } } }
I20260808 07:15:10.252055 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.249965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57602 (local address 127.22.2.130:44317)
0808 07:15:10.250004 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.250006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.250033 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:15:10.250095 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.250096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.250113 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.250172 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:10.250273 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.250701 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.251383 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.251595 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.251658 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.251787 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:10.251871 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:10.251938 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.251981 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.251989 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:10.253687 10931 raft_consensus.cc:1261] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 1 FOLLOWER]: Refusing update from remote peer bf34b885de6d430ab5664b2b01b9f47e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:10.253733 10631 raft_consensus.cc:1261] T 0c5d2071443c4944a70dccb2f1c838f4 P b0a2049231364843ba944b91f1a725d3 [term 1 FOLLOWER]: Refusing update from remote peer bf34b885de6d430ab5664b2b01b9f47e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:10.253985 11007 consensus_queue.cc:1035] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d19695e22e04abc940316b5aa57271d" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:10.254124 11013 consensus_queue.cc:1035] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a2049231364843ba944b91f1a725d3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:10.256572 11017 mvcc.cc:205] Tried to move back new op lower bound from 7316165878796472320 to 7316165878757928960. Current Snapshot: MvccSnapshot[applied={T|T < 7316165878796472320}]
I20260808 07:15:10.256727 11018 mvcc.cc:205] Tried to move back new op lower bound from 7316165878796472320 to 7316165878757928960. Current Snapshot: MvccSnapshot[applied={T|T < 7316165878796472320}]
I20260808 07:15:10.259001 10467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:10.259099 10467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:10.259858 10467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8f8250e5d724b32986474dbf430befa: Sending DeleteTablet for 3 replicas of tablet 0c5d2071443c4944a70dccb2f1c838f4
I20260808 07:15:10.260147 10762 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c5d2071443c4944a70dccb2f1c838f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:10 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:57594
I20260808 07:15:10.260185 10607 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c5d2071443c4944a70dccb2f1c838f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:10 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:49646
I20260808 07:15:10.260236 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c5d2071443c4944a70dccb2f1c838f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:10 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:60640
I20260808 07:15:10.260442 11027 tablet_replica.cc:330] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: stopping tablet replica
I20260808 07:15:10.260615 11029 tablet_replica.cc:330] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: stopping tablet replica
I20260808 07:15:10.260708 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 10542
I20260808 07:15:10.260725 11029 raft_consensus.cc:2229] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:10.260836 11029 raft_consensus.cc:2258] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:10.263767 11029 ts_tablet_manager.cc:1905] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:10.265576 11029 ts_tablet_manager.cc:1918] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:10.265659 11029 log.cc:1196] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/0c5d2071443c4944a70dccb2f1c838f4
I20260808 07:15:10.265923 11029 ts_tablet_manager.cc:1939] T 0c5d2071443c4944a70dccb2f1c838f4 P 6d19695e22e04abc940316b5aa57271d: Deleting consensus metadata
I20260808 07:15:10.260537 11027 raft_consensus.cc:2229] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:15:10.266301 10451 catalog_manager.cc:4841] TS 6d19695e22e04abc940316b5aa57271d (127.22.2.131:41487): tablet 0c5d2071443c4944a70dccb2f1c838f4 (table test-table [id=83e12fb165574a04871f9b20ca529ea9]) successfully deleted
I20260808 07:15:10.266398 11027 raft_consensus.cc:2258] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:10.266829 11027 ts_tablet_manager.cc:1905] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:10.268602 11027 ts_tablet_manager.cc:1918] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:10.268684 11027 log.cc:1196] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/0c5d2071443c4944a70dccb2f1c838f4
I20260808 07:15:10.268970 11027 ts_tablet_manager.cc:1939] T 0c5d2071443c4944a70dccb2f1c838f4 P bf34b885de6d430ab5664b2b01b9f47e: Deleting consensus metadata
I20260808 07:15:10.269415 10451 catalog_manager.cc:4841] TS bf34b885de6d430ab5664b2b01b9f47e (127.22.2.130:44317): tablet 0c5d2071443c4944a70dccb2f1c838f4 (table test-table [id=83e12fb165574a04871f9b20ca529ea9]) successfully deleted
W20260808 07:15:10.271569 10453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:37765 (error 108)
I20260808 07:15:10.271863 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 10695
I20260808 07:15:10.272140 10998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.272021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37765 (local address 127.0.0.1:49654)
0808 07:15:10.272069 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.272101 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:15:10.272244 10453 catalog_manager.cc:4584] TS b0a2049231364843ba944b91f1a725d3 (127.22.2.129:37765): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c5d2071443c4944a70dccb2f1c838f4: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37765: connect: Connection refused (error 111)
I20260808 07:15:10.279239 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 10842
I20260808 07:15:10.284773 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 10434
2026-08-08T07:15:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2153 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T07:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:10Z Disabled control of system clock
I20260808 07:15:10.315323 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38641
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:38641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:15:10.399842 11034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:10.400099 11034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:10.400123 11034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:10.401255 11034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:10.401301 11034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:10.401316 11034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:10.401332 11034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:10.401346 11034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:10.402691 11034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:38641
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:38641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:10.402911 11034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:10.403168 11034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:10.408326 11040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:10.408335 11042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:10.408329 11039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:10.408649 11034 server_base.cc:1034] running on GCE node
I20260808 07:15:10.408890 11034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:10.409132 11034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:10.410267 11034 hybrid_clock.cc:648] HybridClock initialized: now 1786173310410245 us; error 38 us; skew 500 ppm
I20260808 07:15:10.415678 11034 webserver.cc:460] Webserver started at http://127.22.2.190:39107/ using document root <none> and password file <none>
I20260808 07:15:10.415899 11034 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:10.415942 11034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:10.416038 11034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:10.417094 11034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "7f24ba57ae2d4fc192b447689581d759"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "cbbfb36857b0690844dd10b274ff6322"
server_key_iv: "c543fbc3adae5e45f9f40f7042b7b0c7"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.417403 11034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "7f24ba57ae2d4fc192b447689581d759"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "cbbfb36857b0690844dd10b274ff6322"
server_key_iv: "c543fbc3adae5e45f9f40f7042b7b0c7"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.418640 11034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:10.419317 11049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.419505 11034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:10.419617 11034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "7f24ba57ae2d4fc192b447689581d759"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "cbbfb36857b0690844dd10b274ff6322"
server_key_iv: "c543fbc3adae5e45f9f40f7042b7b0c7"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.419682 11034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:10.432719 11034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:10.432992 11034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:10.433099 11034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:10.437215 11034 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:38641
I20260808 07:15:10.437251 11107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:38641 every 8 connection(s)
I20260808 07:15:10.437637 11034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:10.438835 11108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:10.439939 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 11034
I20260808 07:15:10.440053 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:10.440284 22538 external_mini_cluster.cc:1428] Setting key e19599427d9a43226ef73a985ed54908
I20260808 07:15:10.441813 11108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759: Bootstrap starting.
I20260808 07:15:10.442366 11108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:10.442620 11108 log.cc:824] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:10.443286 11108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759: No bootstrap required, opened a new log
I20260808 07:15:10.445160 11108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f24ba57ae2d4fc192b447689581d759" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38641 } }
I20260808 07:15:10.445319 11108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:10.445393 11108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f24ba57ae2d4fc192b447689581d759, State: Initialized, Role: FOLLOWER
I20260808 07:15:10.445556 11108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f24ba57ae2d4fc192b447689581d759" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38641 } }
I20260808 07:15:10.445647 11108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:10.445693 11108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:10.445741 11108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:10.446280 11108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f24ba57ae2d4fc192b447689581d759" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38641 } }
I20260808 07:15:10.446396 11108 leader_election.cc:302] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f24ba57ae2d4fc192b447689581d759; no voters:
I20260808 07:15:10.446591 11108 leader_election.cc:288] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:10.446727 11113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:10.446897 11113 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [term 1 LEADER]: Becoming Leader. State: Replica: 7f24ba57ae2d4fc192b447689581d759, State: Running, Role: LEADER
I20260808 07:15:10.446972 11108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:10.447002 11113 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f24ba57ae2d4fc192b447689581d759" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38641 } }
I20260808 07:15:10.447345 11113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f24ba57ae2d4fc192b447689581d759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f24ba57ae2d4fc192b447689581d759" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38641 } } }
I20260808 07:15:10.447429 11113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:10.447618 11113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f24ba57ae2d4fc192b447689581d759. Latest consensus state: current_term: 1 leader_uuid: "7f24ba57ae2d4fc192b447689581d759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f24ba57ae2d4fc192b447689581d759" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 38641 } } }
I20260808 07:15:10.447716 11113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:10.447822 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.440842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.22.2.190:38641)
0808 07:15:10.442848 (+ 2006us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.442855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.442879 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:15:10.443280 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.443287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.443340 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.443506 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:10.443779 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.444688 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.445532 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.446037 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.446176 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.446956 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:10.446961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:10.446961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:10.447182 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:10.447226 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:10.447240 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:10.447308 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.447404 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.447496 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1944,"thread_start_us":66,"threads_started":1}
I20260808 07:15:10.447851 11119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:10.448338 11119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:10.449750 11119 catalog_manager.cc:1347] Generated new cluster ID: 59330f9d6b164ff18c5e0e18b19c0760
I20260808 07:15:10.449795 11119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:10.467944 11119 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:10.468511 11119 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:10.474632 11119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759: Generated new TSK 0
I20260808 07:15:10.474855 11119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:10.478159 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38641
--builtin_ntp_servers=127.22.2.148:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:15:10.572378 11134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:10.572629 11134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:10.572695 11134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:10.574052 11134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:10.574120 11134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:10.574167 11134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:15:10.575832 11134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:10.576123 11134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:10.576457 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:10.582522 11140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:10.582628 11142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:10.582528 11139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:10.582979 11134 server_base.cc:1034] running on GCE node
I20260808 07:15:10.583218 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:10.583523 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:10.584723 11134 hybrid_clock.cc:648] HybridClock initialized: now 1786173310584706 us; error 30 us; skew 500 ppm
I20260808 07:15:10.586127 11134 webserver.cc:460] Webserver started at http://127.22.2.129:36161/ using document root <none> and password file <none>
I20260808 07:15:10.586439 11134 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:10.586517 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:10.586690 11134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:10.587977 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "9505c8634ce1480b8dde69c68bbb2c5c"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "3a85101796939e732d1153b4535bdd34"
server_key_iv: "d38290f187559cc1966df75eb95f3317"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.588370 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "9505c8634ce1480b8dde69c68bbb2c5c"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "3a85101796939e732d1153b4535bdd34"
server_key_iv: "d38290f187559cc1966df75eb95f3317"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.589795 11134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:15:10.590534 11148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.590698 11134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:10.590801 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "9505c8634ce1480b8dde69c68bbb2c5c"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "3a85101796939e732d1153b4535bdd34"
server_key_iv: "d38290f187559cc1966df75eb95f3317"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.590884 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:10.599618 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:10.599893 11134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:10.600032 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:10.600242 11134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:10.600574 11134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:10.600626 11134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.600667 11134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:10.600696 11134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.606561 11134 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:38323
I20260808 07:15:10.606633 11275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:38323 every 8 connection(s)
I20260808 07:15:10.606940 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:15:10.612099 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.607640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34037 (local address 127.22.2.190:38641)
0808 07:15:10.607683 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.607686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.607938 (+ 252us) server_negotiation.cc:407] Connection header received
0808 07:15:10.608457 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.608459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.608472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.608535 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:10.609194 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.609692 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.610681 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.610907 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.610979 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.611807 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:10.611808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:10.611809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:10.611814 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:10.611829 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:10.611841 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:10.611907 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.611995 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.612021 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:10.612303 11278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.607641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38641 (local address 127.22.2.129:34037)
0808 07:15:10.607821 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.607855 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.608358 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.608606 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.608610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.608617 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:10.609105 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.609110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.609797 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.609798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.610589 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.610591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.611000 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.611001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.611122 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.611652 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:10.611653 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:10.611680 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:10.611702 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:10.611907 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:10.611914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:10.611918 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.611997 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.612094 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260808 07:15:10.612787 11276 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38641
I20260808 07:15:10.612887 11276 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:10.613057 11276 heartbeater.cc:511] Master 127.22.2.190:38641 requested a full tablet report, sending...
I20260808 07:15:10.613543 11067 ts_manager.cc:194] Registered new tserver with Master: 9505c8634ce1480b8dde69c68bbb2c5c (127.22.2.129:38323)
I20260808 07:15:10.614042 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 11134
I20260808 07:15:10.614146 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:15:10.614269 11067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.129:34037
I20260808 07:15:10.614337 22538 external_mini_cluster.cc:1428] Setting key 10af3a3dbcb9b459073b799e7971f71e
I20260808 07:15:10.615211 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38641
--builtin_ntp_servers=127.22.2.148:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:15:10.699297 11279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:10.699522 11279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:10.699544 11279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:10.700744 11279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:10.700789 11279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:10.700815 11279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:15:10.702147 11279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:10.702396 11279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:10.702682 11279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:10.707743 11288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:10.707743 11286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:10.707808 11279 server_base.cc:1034] running on GCE node
W20260808 07:15:10.707767 11285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:10.708158 11279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:10.708362 11279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:10.709496 11279 hybrid_clock.cc:648] HybridClock initialized: now 1786173310709486 us; error 30 us; skew 500 ppm
I20260808 07:15:10.710704 11279 webserver.cc:460] Webserver started at http://127.22.2.130:34419/ using document root <none> and password file <none>
I20260808 07:15:10.710906 11279 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:10.710947 11279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:10.711035 11279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:10.712122 11279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "de40cc577daf4074aaf278a261f031ac"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "d48b055964a1fdee4fd44f6cec500e34"
server_key_iv: "4b3d822fa0e8390789bba029e2f63c92"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.712435 11279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "de40cc577daf4074aaf278a261f031ac"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "d48b055964a1fdee4fd44f6cec500e34"
server_key_iv: "4b3d822fa0e8390789bba029e2f63c92"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.713661 11279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:10.714321 11296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.714493 11279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:10.714572 11279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "de40cc577daf4074aaf278a261f031ac"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "d48b055964a1fdee4fd44f6cec500e34"
server_key_iv: "4b3d822fa0e8390789bba029e2f63c92"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.714625 11279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:10.723230 11279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:10.723491 11279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:10.723670 11279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:10.723863 11279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:10.724159 11279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:10.724198 11279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.724220 11279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:10.724233 11279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.729846 11279 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:33545
I20260808 07:15:10.729928 11420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:33545 every 8 connection(s)
I20260808 07:15:10.730242 11279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:15:10.730461 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 11279
I20260808 07:15:10.730558 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:15:10.730790 22538 external_mini_cluster.cc:1428] Setting key fea12f734e8bd7c465fe6546c67a241e
I20260808 07:15:10.731953 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38641
--builtin_ntp_servers=127.22.2.148:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:15:10.736363 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.731255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:53597 (local address 127.22.2.190:38641)
0808 07:15:10.731301 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.731304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.731628 (+ 324us) server_negotiation.cc:407] Connection header received
0808 07:15:10.732388 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.732389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.732400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.732443 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:10.733374 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.733887 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.734901 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.735099 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.735182 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.736070 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:10.736072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:10.736072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:10.736077 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:10.736091 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:10.736095 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:10.736144 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.736254 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.736280 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:10.736593 11424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.731263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38641 (local address 127.22.2.130:53597)
0808 07:15:10.731494 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.731536 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.732290 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.732513 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.732517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.732528 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:10.733272 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.733280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.733995 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.733997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.734808 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.734811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.735196 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.735197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.735335 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.735904 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:10.735905 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:10.735947 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:10.735980 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:10.736149 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:10.736154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:10.736158 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.736254 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.736365 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":127,"threads_started":1}
I20260808 07:15:10.737085 11421 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38641
I20260808 07:15:10.737188 11421 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:10.737391 11421 heartbeater.cc:511] Master 127.22.2.190:38641 requested a full tablet report, sending...
I20260808 07:15:10.737819 11067 ts_manager.cc:194] Registered new tserver with Master: de40cc577daf4074aaf278a261f031ac (127.22.2.130:33545)
I20260808 07:15:10.738185 11067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.130:53597
W20260808 07:15:10.814965 11425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:10.815174 11425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:10.815196 11425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:10.816489 11425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:10.816531 11425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:10.816578 11425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:15:10.818015 11425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:38641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:10.818288 11425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:10.818603 11425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:10.823848 11431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:10.823860 11432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:10.823869 11434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:10.823892 11425 server_base.cc:1034] running on GCE node
I20260808 07:15:10.824290 11425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:10.824534 11425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:10.825754 11425 hybrid_clock.cc:648] HybridClock initialized: now 1786173310825733 us; error 46 us; skew 500 ppm
I20260808 07:15:10.827126 11425 webserver.cc:460] Webserver started at http://127.22.2.131:33767/ using document root <none> and password file <none>
I20260808 07:15:10.827374 11425 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:10.827447 11425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:10.827647 11425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:10.828822 11425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "80a68d35181f44f8998bc6083bc79755"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "b726d49ac6124afcb27a4b2e5d38a70d"
server_key_iv: "85ccf695ab17edd487af9b84bcea7eba"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.829217 11425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "80a68d35181f44f8998bc6083bc79755"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "b726d49ac6124afcb27a4b2e5d38a70d"
server_key_iv: "85ccf695ab17edd487af9b84bcea7eba"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.830581 11425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:10.831295 11440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.831470 11425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:10.831600 11425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "80a68d35181f44f8998bc6083bc79755"
format_stamp: "Formatted at 2026-08-08 07:15:10 on dist-test-slave-6k22"
server_key: "b726d49ac6124afcb27a4b2e5d38a70d"
server_key_iv: "85ccf695ab17edd487af9b84bcea7eba"
server_key_version: "encryptionkey@0"
I20260808 07:15:10.831692 11425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:10.854444 11425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:10.854789 11425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:10.854955 11425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:10.855199 11425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:10.855541 11425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:10.855660 11425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.855703 11425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:10.855731 11425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:10.861929 11425 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:38853
I20260808 07:15:10.861974 11565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:38853 every 8 connection(s)
I20260808 07:15:10.862294 11425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:15:10.867714 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.863065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:47509 (local address 127.22.2.190:38641)
0808 07:15:10.863123 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.863125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.863324 (+ 199us) server_negotiation.cc:407] Connection header received
0808 07:15:10.863869 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.863870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.863882 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.863953 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:10.864613 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.865248 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.866208 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.866451 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.866536 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.867330 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:10.867331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:10.867332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:10.867337 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:10.867355 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:10.867359 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:10.867414 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.867499 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.867525 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:15:10.867712 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 11425
I20260808 07:15:10.867827 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:15:10.867889 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.863012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:38641 (local address 127.22.2.131:47509)
0808 07:15:10.863194 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.863235 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.863773 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.864007 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.864011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.864019 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:10.864532 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.864537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.865332 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.865333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.866098 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.866101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.866543 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.866544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.866649 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.867185 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:10.867185 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:10.867212 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:10.867233 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:10.867421 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:10.867425 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:10.867428 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.867480 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.867631 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260808 07:15:10.868037 22538 external_mini_cluster.cc:1428] Setting key 9d0cfeb0ec3860d69850610477128d27
I20260808 07:15:10.868482 11566 heartbeater.cc:347] Connected to a master server at 127.22.2.190:38641
I20260808 07:15:10.868582 11566 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:10.868774 11566 heartbeater.cc:511] Master 127.22.2.190:38641 requested a full tablet report, sending...
I20260808 07:15:10.869189 11067 ts_manager.cc:194] Registered new tserver with Master: 80a68d35181f44f8998bc6083bc79755 (127.22.2.131:38853)
I20260808 07:15:10.869554 11067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.131:47509
I20260808 07:15:10.880072 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:15:10.884645 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.881565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33124 (local address 127.22.2.190:38641)
0808 07:15:10.881635 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.881637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.881709 (+ 72us) server_negotiation.cc:407] Connection header received
0808 07:15:10.881764 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.881764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.881774 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.881820 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:10.881953 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.882416 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.883275 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.883456 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.883504 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.884303 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:10.884304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:10.884305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:10.884309 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:10.884322 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:10.884325 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:10.884413 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.884512 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.884541 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:15:10.886319 11067 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:15:10.887975 11067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:15:10.894683 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.890078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:33545 (local address 127.0.0.1:53692)
0808 07:15:10.890927 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.890937 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.890984 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.891277 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.891278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.891280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:10.891318 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.891320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.892572 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.892573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.893559 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.893561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.894345 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.894346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.894432 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.894447 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.894521 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.894578 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260808 07:15:10.894804 11578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.890075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:38323 (local address 127.0.0.1:51908)
0808 07:15:10.890749 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.890758 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.890836 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.891079 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.891080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.891083 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:10.891134 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.891136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.892474 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.892475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.893675 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.893677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.894522 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.894523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.894582 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.894586 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.894651 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.894685 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260808 07:15:10.894896 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.890073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38853 (local address 127.0.0.1:38234)
0808 07:15:10.890387 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:10.890429 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:10.890499 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:10.890845 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:10.890847 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:10.890850 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:10.890956 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.890959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.892108 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.892109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.893435 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:10.893438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.894607 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.894608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:10.894716 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.894722 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:15:10.894774 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:15:10.894829 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":93,"threads_started":1}
I20260808 07:15:10.895042 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.890297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.22.2.130:33545)
0808 07:15:10.890511 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.890517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.891052 (+ 535us) server_negotiation.cc:407] Connection header received
0808 07:15:10.891103 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.891106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.891122 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.891198 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:10.891406 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.892487 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.893658 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.894231 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.894327 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.894428 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.894901 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.894952 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":115,"threads_started":1}
I20260808 07:15:10.895217 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.890377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.22.2.129:38323)
0808 07:15:10.890602 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.890605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.890858 (+ 253us) server_negotiation.cc:407] Connection header received
0808 07:15:10.890907 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.890909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.890924 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.891002 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:10.891220 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.892386 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.893757 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.894448 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.894506 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.894579 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.894662 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.894701 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":116,"threads_started":1}
I20260808 07:15:10.895231 11582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:10.890530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.22.2.131:38853)
0808 07:15:10.890685 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:10.890688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:10.890696 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:10.890728 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:10.890730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:10.890742 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:10.890811 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:10.891055 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.892023 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.893518 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:10.894409 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:10.894481 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:10.894568 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:10.895064 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 07:15:10.895120 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":141,"threads_started":1}
I20260808 07:15:10.896057 11202 tablet_service.cc:1467] Processing CreateTablet for tablet 1e7d4d5355c14cd983e1bf3947f8a59e (DEFAULT_TABLE table=test-table [id=606c78cff0334401932a1e05d98fe957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:10.896150 11494 tablet_service.cc:1467] Processing CreateTablet for tablet 1e7d4d5355c14cd983e1bf3947f8a59e (DEFAULT_TABLE table=test-table [id=606c78cff0334401932a1e05d98fe957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:10.896425 11494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e7d4d5355c14cd983e1bf3947f8a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:10.896423 11202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e7d4d5355c14cd983e1bf3947f8a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:10.898669 11348 tablet_service.cc:1467] Processing CreateTablet for tablet 1e7d4d5355c14cd983e1bf3947f8a59e (DEFAULT_TABLE table=test-table [id=606c78cff0334401932a1e05d98fe957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:10.899020 11348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e7d4d5355c14cd983e1bf3947f8a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:10.899540 11583 tablet_bootstrap.cc:492] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Bootstrap starting.
I20260808 07:15:10.899794 11584 tablet_bootstrap.cc:492] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c: Bootstrap starting.
I20260808 07:15:10.900396 11583 tablet_bootstrap.cc:654] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:10.900485 11584 tablet_bootstrap.cc:654] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:10.900797 11584 log.cc:824] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:10.900875 11583 log.cc:824] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:10.901752 11583 tablet_bootstrap.cc:492] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: No bootstrap required, opened a new log
I20260808 07:15:10.901746 11584 tablet_bootstrap.cc:492] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c: No bootstrap required, opened a new log
I20260808 07:15:10.901835 11584 ts_tablet_manager.cc:1397] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:15:10.901975 11583 ts_tablet_manager.cc:1397] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:15:10.903430 11588 tablet_bootstrap.cc:492] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: Bootstrap starting.
I20260808 07:15:10.904206 11588 tablet_bootstrap.cc:654] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:10.904587 11588 log.cc:824] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:10.904780 11583 raft_consensus.cc:348] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:10.904803 11584 raft_consensus.cc:348] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:10.904978 11583 raft_consensus.cc:374] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:10.905005 11584 raft_consensus.cc:374] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:10.905030 11583 raft_consensus.cc:729] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a68d35181f44f8998bc6083bc79755, State: Initialized, Role: FOLLOWER
I20260808 07:15:10.905058 11584 raft_consensus.cc:729] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9505c8634ce1480b8dde69c68bbb2c5c, State: Initialized, Role: FOLLOWER
I20260808 07:15:10.905159 11583 consensus_queue.cc:260] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:10.905222 11584 consensus_queue.cc:260] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:10.905462 11583 ts_tablet_manager.cc:1428] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:15:10.905530 11566 heartbeater.cc:503] Master 127.22.2.190:38641 was elected leader, sending a full tablet report...
I20260808 07:15:10.905807 11588 tablet_bootstrap.cc:492] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: No bootstrap required, opened a new log
I20260808 07:15:10.906008 11584 ts_tablet_manager.cc:1428] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:15:10.906155 11276 heartbeater.cc:503] Master 127.22.2.190:38641 was elected leader, sending a full tablet report...
I20260808 07:15:10.906435 11588 ts_tablet_manager.cc:1397] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:15:10.909574 11588 raft_consensus.cc:348] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:10.909759 11588 raft_consensus.cc:374] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:10.909804 11588 raft_consensus.cc:729] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de40cc577daf4074aaf278a261f031ac, State: Initialized, Role: FOLLOWER
I20260808 07:15:10.909927 11588 consensus_queue.cc:260] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:10.910225 11588 ts_tablet_manager.cc:1428] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:15:10.910364 11421 heartbeater.cc:503] Master 127.22.2.190:38641 was elected leader, sending a full tablet report...
W20260808 07:15:10.980801 11422 tablet.cc:2369] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:15:11.107730 11277 tablet.cc:2369] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:15:11.112802 11568 tablet.cc:2369] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:15:11.222160 11591 raft_consensus.cc:482] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:15:11.222321 11591 raft_consensus.cc:504] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:11.222757 11591 leader_election.cc:288] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80a68d35181f44f8998bc6083bc79755 (127.22.2.131:38853), de40cc577daf4074aaf278a261f031ac (127.22.2.130:33545)
I20260808 07:15:11.225747 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.223010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:59189 (local address 127.22.2.130:33545)
0808 07:15:11.223069 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.223071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.223241 (+ 170us) server_negotiation.cc:407] Connection header received
0808 07:15:11.223328 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.223329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.223337 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.223387 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:11.223590 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.224044 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.225047 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.225407 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.225490 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.225498 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.225644 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.225680 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:15:11.225764 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.222949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:33545 (local address 127.22.2.129:59189)
0808 07:15:11.223174 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.223182 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.223255 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.223399 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.223399 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.223401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:11.223497 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.223500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.224124 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.224126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.224958 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.224960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.225489 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.225490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.225562 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.225566 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.225630 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.225671 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":46,"thread_start_us":59,"threads_started":1}
I20260808 07:15:11.226042 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.222949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:38853 (local address 127.22.2.129:33075)
0808 07:15:11.223095 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.223105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.223192 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.223373 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.223374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.223376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:11.223456 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.223473 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.224118 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.224119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.224965 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.224968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.225566 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.225568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.225646 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.225650 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.225716 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.225757 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":63,"thread_start_us":69,"threads_started":1}
I20260808 07:15:11.226083 11582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.223005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:33075 (local address 127.22.2.131:38853)
0808 07:15:11.223057 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.223059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.223176 (+ 117us) server_negotiation.cc:407] Connection header received
0808 07:15:11.223265 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.223266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.223276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.223364 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:11.223546 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.224032 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.225056 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.225488 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.225566 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.225574 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.225744 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.225788 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:15:11.226148 11369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e7d4d5355c14cd983e1bf3947f8a59e" candidate_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de40cc577daf4074aaf278a261f031ac" is_pre_election: true
I20260808 07:15:11.226320 11369 raft_consensus.cc:2454] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9505c8634ce1480b8dde69c68bbb2c5c in term 0.
I20260808 07:15:11.226389 11518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e7d4d5355c14cd983e1bf3947f8a59e" candidate_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80a68d35181f44f8998bc6083bc79755" is_pre_election: true
I20260808 07:15:11.226533 11518 raft_consensus.cc:2454] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9505c8634ce1480b8dde69c68bbb2c5c in term 0.
I20260808 07:15:11.226547 11156 leader_election.cc:302] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9505c8634ce1480b8dde69c68bbb2c5c, de40cc577daf4074aaf278a261f031ac; no voters:
I20260808 07:15:11.226656 11591 raft_consensus.cc:2781] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:15:11.226714 11591 raft_consensus.cc:482] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:15:11.226732 11591 raft_consensus.cc:3037] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:11.227494 11591 raft_consensus.cc:504] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:11.227654 11591 leader_election.cc:288] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [CANDIDATE]: Term 1 election: Requested vote from peers 80a68d35181f44f8998bc6083bc79755 (127.22.2.131:38853), de40cc577daf4074aaf278a261f031ac (127.22.2.130:33545)
I20260808 07:15:11.227828 11518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e7d4d5355c14cd983e1bf3947f8a59e" candidate_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80a68d35181f44f8998bc6083bc79755"
I20260808 07:15:11.227841 11369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e7d4d5355c14cd983e1bf3947f8a59e" candidate_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de40cc577daf4074aaf278a261f031ac"
I20260808 07:15:11.227908 11518 raft_consensus.cc:3037] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:11.227908 11369 raft_consensus.cc:3037] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:11.228822 11518 raft_consensus.cc:2454] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9505c8634ce1480b8dde69c68bbb2c5c in term 1.
I20260808 07:15:11.228833 11369 raft_consensus.cc:2454] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9505c8634ce1480b8dde69c68bbb2c5c in term 1.
I20260808 07:15:11.229014 11152 leader_election.cc:302] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80a68d35181f44f8998bc6083bc79755, 9505c8634ce1480b8dde69c68bbb2c5c; no voters:
I20260808 07:15:11.229115 11591 raft_consensus.cc:2781] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:11.229262 11591 raft_consensus.cc:686] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 1 LEADER]: Becoming Leader. State: Replica: 9505c8634ce1480b8dde69c68bbb2c5c, State: Running, Role: LEADER
I20260808 07:15:11.229367 11591 consensus_queue.cc:237] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } }
I20260808 07:15:11.230182 11066 catalog_manager.cc:5495] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9505c8634ce1480b8dde69c68bbb2c5c (127.22.2.129). New cstate: current_term: 1 leader_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9505c8634ce1480b8dde69c68bbb2c5c" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 38323 } health_report { overall_health: HEALTHY } } }
I20260808 07:15:11.246627 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.244570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51924 (local address 127.22.2.129:38323)
0808 07:15:11.244622 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.244624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.244636 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:15:11.244668 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.244669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.244686 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.244750 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:11.244868 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.245314 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.246014 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.246191 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.246239 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.246373 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:11.246455 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:11.246522 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.246562 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.246571 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:15:11.248548 11518 raft_consensus.cc:1261] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 1 FOLLOWER]: Refusing update from remote peer 9505c8634ce1480b8dde69c68bbb2c5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:11.248548 11369 raft_consensus.cc:1261] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 1 FOLLOWER]: Refusing update from remote peer 9505c8634ce1480b8dde69c68bbb2c5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:11.249059 11591 consensus_queue.cc:1035] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80a68d35181f44f8998bc6083bc79755" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 38853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:11.249217 11595 consensus_queue.cc:1035] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "de40cc577daf4074aaf278a261f031ac" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 33545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:11.251709 11598 mvcc.cc:205] Tried to move back new op lower bound from 7316165882871234560 to 7316165882796064768. Current Snapshot: MvccSnapshot[applied={T|T < 7316165882871234560}]
I20260808 07:15:11.252090 11601 mvcc.cc:205] Tried to move back new op lower bound from 7316165882871234560 to 7316165882796064768. Current Snapshot: MvccSnapshot[applied={T|T < 7316165882871234560}]
I20260808 07:15:11.254158 11067 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:11.254256 11067 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:11.254982 11067 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f24ba57ae2d4fc192b447689581d759: Sending DeleteTablet for 3 replicas of tablet 1e7d4d5355c14cd983e1bf3947f8a59e
I20260808 07:15:11.255324 11494 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e7d4d5355c14cd983e1bf3947f8a59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:11 UTC) from {username='slave'} at 127.0.0.1:38234
I20260808 07:15:11.255323 11202 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e7d4d5355c14cd983e1bf3947f8a59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:11 UTC) from {username='slave'} at 127.0.0.1:51908
I20260808 07:15:11.255605 11610 tablet_replica.cc:330] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c: stopping tablet replica
I20260808 07:15:11.255589 11348 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e7d4d5355c14cd983e1bf3947f8a59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:11 UTC) from {username='slave'} at 127.0.0.1:53692
I20260808 07:15:11.255780 11609 tablet_replica.cc:330] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: stopping tablet replica
I20260808 07:15:11.255771 11610 raft_consensus.cc:2229] T 1e7d4d5355c14cd983e1bf3947f8a59e P 9505c8634ce1480b8dde69c68bbb2c5c [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:15:11.255872 11609 raft_consensus.cc:2229] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:11.255977 11609 raft_consensus.cc:2258] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:11.255992 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 11134
I20260808 07:15:11.258474 11609 ts_tablet_manager.cc:1905] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:11.260867 11609 ts_tablet_manager.cc:1918] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:11.260949 11609 log.cc:1196] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/1e7d4d5355c14cd983e1bf3947f8a59e
I20260808 07:15:11.261178 11609 ts_tablet_manager.cc:1939] T 1e7d4d5355c14cd983e1bf3947f8a59e P 80a68d35181f44f8998bc6083bc79755: Deleting consensus metadata
I20260808 07:15:11.261685 11051 catalog_manager.cc:4841] TS 80a68d35181f44f8998bc6083bc79755 (127.22.2.131:38853): tablet 1e7d4d5355c14cd983e1bf3947f8a59e (table test-table [id=606c78cff0334401932a1e05d98fe957]) successfully deleted
I20260808 07:15:11.263684 11611 tablet_replica.cc:330] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: stopping tablet replica
I20260808 07:15:11.263805 11611 raft_consensus.cc:2229] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:11.263926 11611 raft_consensus.cc:2258] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:11.264319 11611 ts_tablet_manager.cc:1905] T 1e7d4d5355c14cd983e1bf3947f8a59e P de40cc577daf4074aaf278a261f031ac: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:15:11.265260 11050 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:38323 (error 108)
I20260808 07:15:11.265508 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 11279
I20260808 07:15:11.266909 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.265583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:38323 (local address 127.0.0.1:51934)
0808 07:15:11.266838 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.266881 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1231}
W20260808 07:15:11.267062 11050 catalog_manager.cc:4584] TS 9505c8634ce1480b8dde69c68bbb2c5c (127.22.2.129:38323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e7d4d5355c14cd983e1bf3947f8a59e: Network error: Client connection negotiation failed: client connection to 127.22.2.129:38323: connect: Connection refused (error 111)
I20260808 07:15:11.272739 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.272639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:33545 (local address 127.0.0.1:53698)
0808 07:15:11.272687 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.272714 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.130:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:15:11.272778 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 11425
I20260808 07:15:11.279445 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 11034
2026-08-08T07:15:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (993 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:11 dist-test-slave-6k22 krb5kdc[11615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:11 dist-test-slave-6k22 krb5kdc[11615](info): set up 2 sockets
Aug 08 07:15:11 dist-test-slave-6k22 krb5kdc[11615](info): commencing operation
krb5kdc: starting...
W20260808 07:15:12.419051 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:12 dist-test-slave-6k22 krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:12.545745 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42003
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:12.628010 11628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:12.628218 11628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:12.628240 11628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:12.629390 11628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:12.629433 11628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:12.629448 11628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:12.629493 11628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:12.629518 11628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:12.630985 11628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:12.631230 11628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:12.631458 11628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:15:12.636390 11628 server_base.cc:1034] running on GCE node
W20260808 07:15:12.636363 11633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:12.636371 11636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:12.636407 11634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:12.636919 11628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:12.637166 11628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:12.638305 11628 hybrid_clock.cc:648] HybridClock initialized: now 1786173312638291 us; error 34 us; skew 500 ppm
Aug 08 07:15:12 dist-test-slave-6k22 krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:12.640986 11628 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:12.641412 11628 webserver.cc:460] Webserver started at http://127.22.2.190:36605/ using document root <none> and password file <none>
I20260808 07:15:12.641633 11628 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:12.641696 11628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:12.641847 11628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:12.642889 11628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "377f8cd9773a45ff81cb383fa9896f84"
format_stamp: "Formatted at 2026-08-08 07:15:12 on dist-test-slave-6k22"
server_key: "e626295162e0ace6423076fd2da0e628"
server_key_iv: "19c7975261b32154c93f7925cee07fac"
server_key_version: "encryptionkey@0"
I20260808 07:15:12.643303 11628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "377f8cd9773a45ff81cb383fa9896f84"
format_stamp: "Formatted at 2026-08-08 07:15:12 on dist-test-slave-6k22"
server_key: "e626295162e0ace6423076fd2da0e628"
server_key_iv: "19c7975261b32154c93f7925cee07fac"
server_key_version: "encryptionkey@0"
I20260808 07:15:12.644667 11628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:12.645376 11645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:12.645547 11628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:12.645641 11628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "377f8cd9773a45ff81cb383fa9896f84"
format_stamp: "Formatted at 2026-08-08 07:15:12 on dist-test-slave-6k22"
server_key: "e626295162e0ace6423076fd2da0e628"
server_key_iv: "19c7975261b32154c93f7925cee07fac"
server_key_version: "encryptionkey@0"
I20260808 07:15:12.645721 11628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:12.673655 11628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:12.676126 11628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:12.676296 11628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:12.680419 11628 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42003
I20260808 07:15:12.680485 11701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42003 every 8 connection(s)
I20260808 07:15:12.680894 11628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:12.681936 11703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:12.684515 11703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84: Bootstrap starting.
I20260808 07:15:12.685135 11703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:12.685499 11703 log.cc:824] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:12.686278 11703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84: No bootstrap required, opened a new log
I20260808 07:15:12.688336 11703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377f8cd9773a45ff81cb383fa9896f84" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42003 } }
I20260808 07:15:12.688485 11703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:12.688532 11703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 377f8cd9773a45ff81cb383fa9896f84, State: Initialized, Role: FOLLOWER
I20260808 07:15:12.688644 11703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377f8cd9773a45ff81cb383fa9896f84" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42003 } }
I20260808 07:15:12.688752 11703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:12.688776 11703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:12.688812 11703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:12.689389 11703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377f8cd9773a45ff81cb383fa9896f84" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42003 } }
I20260808 07:15:12.689478 11703 leader_election.cc:302] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 377f8cd9773a45ff81cb383fa9896f84; no voters:
I20260808 07:15:12.689620 11703 leader_election.cc:288] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:12.689679 11706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:12.689855 11706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [term 1 LEADER]: Becoming Leader. State: Replica: 377f8cd9773a45ff81cb383fa9896f84, State: Running, Role: LEADER
I20260808 07:15:12.689881 11703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:12.689949 11706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377f8cd9773a45ff81cb383fa9896f84" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42003 } }
I20260808 07:15:12.690526 11708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 377f8cd9773a45ff81cb383fa9896f84. Latest consensus state: current_term: 1 leader_uuid: "377f8cd9773a45ff81cb383fa9896f84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377f8cd9773a45ff81cb383fa9896f84" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42003 } } }
I20260808 07:15:12.690619 11708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:12.690660 11707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "377f8cd9773a45ff81cb383fa9896f84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377f8cd9773a45ff81cb383fa9896f84" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42003 } } }
I20260808 07:15:12.690763 11707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:12.691099 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 11628
I20260808 07:15:12.691208 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:12.691394 22538 external_mini_cluster.cc:1428] Setting key cc0c037b48ca86cc681a5cd7078acc02
W20260808 07:15:12.692703 11726 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:12.692771 11726 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:12.692833 11715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:12.692996 11715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:12.694484 11715 catalog_manager.cc:1347] Generated new cluster ID: 9f498d15cdc8452bbdeb89c7fe3e334b
I20260808 07:15:12.694540 11715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:12 dist-test-slave-6k22 krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:12.702432 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:12.692044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.22.2.190:42003)
0808 07:15:12.692326 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:12.692335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:12.692378 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:15:12.692470 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:12.692475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:12.692517 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:12.692672 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:12.693427 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.694820 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.695588 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.696026 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.696151 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:12.698512 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:12.698516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:12.698517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:12.698529 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:12.700835 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:12.701253 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:12.701255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:12.701256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:12.701349 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:12.701553 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:12.701554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:12.701554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:12.701797 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:12.701961 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:12.702069 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 07:15:12.702223 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260808 07:15:12.709933 11715 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:12.710680 11715 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:12.717869 11715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84: Generated new TSK 0
I20260808 07:15:12.718053 11715 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.2.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
Entry for principal kudu/127.22.2.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab.
I20260808 07:15:12.749894 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42003
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:12.833379 11732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:12.833585 11732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:12.833607 11732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:12.834757 11732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:12.834796 11732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:12.834822 11732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:15:12.836289 11732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.129.keytab
--principal=kudu/127.22.2.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:12.836489 11732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:12.836671 11732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:12.841704 11737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:12.841765 11738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:12.841957 11732 server_base.cc:1034] running on GCE node
W20260808 07:15:12.841745 11741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:12.842218 11732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:12.842461 11732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:12.843618 11732 hybrid_clock.cc:648] HybridClock initialized: now 1786173312843598 us; error 37 us; skew 500 ppm
Aug 08 07:15:12 dist-test-slave-6k22 krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:12.846148 11732 init.cc:377] Logged in from keytab as kudu/127.22.2.129@KRBTEST.COM (short username kudu)
I20260808 07:15:12.846546 11732 webserver.cc:460] Webserver started at http://127.22.2.129:36231/ using document root <none> and password file <none>
I20260808 07:15:12.846732 11732 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:12.846772 11732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:12.846861 11732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:12.847976 11732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "2361b847ff7b41d0ab44ddecfb39b28f"
format_stamp: "Formatted at 2026-08-08 07:15:12 on dist-test-slave-6k22"
server_key: "8c8896002ceb93c624d1abbcadfbd11f"
server_key_iv: "e521ad6985cd3b43718301c2afcaafda"
server_key_version: "encryptionkey@0"
I20260808 07:15:12.848378 11732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "2361b847ff7b41d0ab44ddecfb39b28f"
format_stamp: "Formatted at 2026-08-08 07:15:12 on dist-test-slave-6k22"
server_key: "8c8896002ceb93c624d1abbcadfbd11f"
server_key_iv: "e521ad6985cd3b43718301c2afcaafda"
server_key_version: "encryptionkey@0"
I20260808 07:15:12.849735 11732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:12.850455 11752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:12.850642 11732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:12.850776 11732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "2361b847ff7b41d0ab44ddecfb39b28f"
format_stamp: "Formatted at 2026-08-08 07:15:12 on dist-test-slave-6k22"
server_key: "8c8896002ceb93c624d1abbcadfbd11f"
server_key_iv: "e521ad6985cd3b43718301c2afcaafda"
server_key_version: "encryptionkey@0"
I20260808 07:15:12.850844 11732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:12.862259 11732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:12.864172 11732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:12.864317 11732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:12.864539 11732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:12.864866 11732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:12.864919 11732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:12.864960 11732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:12.864985 11732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:12.870736 11732 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:32963
I20260808 07:15:12.870813 11873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:32963 every 8 connection(s)
I20260808 07:15:12.871167 11732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:15:12.875334 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 11732
I20260808 07:15:12.875453 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:15:12.875684 22538 external_mini_cluster.cc:1428] Setting key a6a2bc2a06c1b9ec0efb819687d1fb35
Aug 08 07:15:12 dist-test-slave-6k22 krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.129@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:12.881105 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:12.871966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:38769 (local address 127.22.2.190:42003)
0808 07:15:12.872048 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:12.872051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:12.872242 (+ 191us) server_negotiation.cc:407] Connection header received
0808 07:15:12.872775 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:12.872776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:12.872789 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:12.872838 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:12.873798 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.874305 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.875316 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.875651 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.875725 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:12.877974 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:12.877976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:12.877977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:12.877984 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:12.879611 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:12.880121 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:12.880123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:12.880123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:12.880188 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:12.880475 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:12.880476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:12.880476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:12.880585 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:12.880637 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:12.880968 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 07:15:12.881040 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:15:12.881436 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:12.871942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42003 (local address 127.22.2.129:38769)
0808 07:15:12.872119 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:12.872156 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:12.872667 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:12.872933 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:12.872937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:12.873502 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:12.873693 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:12.873697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.874410 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.874412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:12.875206 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:12.875208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.875734 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.875736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:12.875868 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:12.876507 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:12.876516 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:12.877877 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:12.879707 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:12.879716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:12.879719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:12.880045 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:12.880257 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:12.880258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:12.880258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:12.880407 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:12.880645 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:12.880652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:12.880821 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 07:15:12.880946 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:15:12.881148 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260808 07:15:12.882046 11874 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42003
I20260808 07:15:12.882157 11874 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:12.882366 11874 heartbeater.cc:511] Master 127.22.2.190:42003 requested a full tablet report, sending...
I20260808 07:15:12.882894 11663 ts_manager.cc:194] Registered new tserver with Master: 2361b847ff7b41d0ab44ddecfb39b28f (127.22.2.129:32963)
I20260808 07:15:12.883505 11663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.129@KRBTEST.COM'} at 127.22.2.129:38769
WARNING: no policy specified for kudu/127.22.2.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
Entry for principal kudu/127.22.2.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab.
I20260808 07:15:12.906164 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42003
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:12.990830 11879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:12.991074 11879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:12.991117 11879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:12.992419 11879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:12.992481 11879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:12.992527 11879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:15:12.993985 11879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.130.keytab
--principal=kudu/127.22.2.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:12.994257 11879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:12.994501 11879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:12.999534 11889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:12.999670 11879 server_base.cc:1034] running on GCE node
W20260808 07:15:12.999542 11885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:12.999547 11884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:13.000036 11879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:13.000303 11879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:13.001451 11879 hybrid_clock.cc:648] HybridClock initialized: now 1786173313001434 us; error 28 us; skew 500 ppm
Aug 08 07:15:12 dist-test-slave-6k22 krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:13.003911 11879 init.cc:377] Logged in from keytab as kudu/127.22.2.130@KRBTEST.COM (short username kudu)
I20260808 07:15:13.004347 11879 webserver.cc:460] Webserver started at http://127.22.2.130:42501/ using document root <none> and password file <none>
I20260808 07:15:13.004530 11879 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:13.004571 11879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:13.004660 11879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:13.005663 11879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "39275a9b80bc4fd7830334b7eca3dab7"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "7f9ec850112fc5c38370f12cb0c65c7c"
server_key_iv: "4b77d9dea3f934b8e78a6ccd99db0a4d"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.005990 11879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "39275a9b80bc4fd7830334b7eca3dab7"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "7f9ec850112fc5c38370f12cb0c65c7c"
server_key_iv: "4b77d9dea3f934b8e78a6ccd99db0a4d"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.007246 11879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:15:13.007983 11896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.008152 11879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.008236 11879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "39275a9b80bc4fd7830334b7eca3dab7"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "7f9ec850112fc5c38370f12cb0c65c7c"
server_key_iv: "4b77d9dea3f934b8e78a6ccd99db0a4d"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.008291 11879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:13.033425 11879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:13.035823 11879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:13.035941 11879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:13.036145 11879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:13.036489 11879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:13.036526 11879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.036569 11879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:13.036597 11879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.042513 11879 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:38567
I20260808 07:15:13.042608 12016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:38567 every 8 connection(s)
I20260808 07:15:13.042917 11879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:15:13 dist-test-slave-6k22 krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173312, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.130@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:13.051823 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.043668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:44335 (local address 127.22.2.190:42003)
0808 07:15:13.043749 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.043752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.043989 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:15:13.044470 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.044471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.044482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.044535 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:13.045453 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.045945 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.046895 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.047083 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.047136 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.049432 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:13.049434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:13.049434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:13.049441 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:13.050661 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:13.051014 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:13.051015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:13.051015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:13.051065 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:13.051262 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:13.051263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:13.051263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:13.051374 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:13.051424 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.051691 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.051757 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:15:13.052000 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.043677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42003 (local address 127.22.2.130:44335)
0808 07:15:13.043860 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.043898 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:13.044387 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:13.044590 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:13.044593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:13.045175 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:13.045375 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.045380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.046022 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.046023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.046815 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.046817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.047156 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.047156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.047255 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.047800 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:13.047808 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:13.049347 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:13.050730 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:13.050736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:13.050738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:13.050953 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:13.051119 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:13.051119 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:13.051120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:13.051213 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:13.051426 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:13.051430 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:13.051541 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 07:15:13.051675 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 07:15:13.051797 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260808 07:15:13.052454 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 11879
I20260808 07:15:13.052484 12017 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42003
I20260808 07:15:13.052585 12017 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:13.052587 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:15:13.052790 12017 heartbeater.cc:511] Master 127.22.2.190:42003 requested a full tablet report, sending...
I20260808 07:15:13.052811 22538 external_mini_cluster.cc:1428] Setting key 55b4e27a3b05efe9a95adb069aec7656
I20260808 07:15:13.053217 11662 ts_manager.cc:194] Registered new tserver with Master: 39275a9b80bc4fd7830334b7eca3dab7 (127.22.2.130:38567)
I20260808 07:15:13.053704 11662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.130@KRBTEST.COM'} at 127.22.2.130:44335
WARNING: no policy specified for kudu/127.22.2.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
Entry for principal kudu/127.22.2.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab.
I20260808 07:15:13.078907 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42003
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:13.166627 12023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:13.166874 12023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:13.166915 12023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:13.168154 12023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:13.168221 12023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:13.168264 12023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:15:13.169724 12023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.131.keytab
--principal=kudu/127.22.2.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:13.169986 12023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:13.170212 12023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:13.175350 12031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:13.175345 12033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:13.175478 12023 server_base.cc:1034] running on GCE node
W20260808 07:15:13.175602 12030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:13.175817 12023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:13.176026 12023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:13.177160 12023 hybrid_clock.cc:648] HybridClock initialized: now 1786173313177138 us; error 36 us; skew 500 ppm
Aug 08 07:15:13 dist-test-slave-6k22 krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173313, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:13.179641 12023 init.cc:377] Logged in from keytab as kudu/127.22.2.131@KRBTEST.COM (short username kudu)
I20260808 07:15:13.180032 12023 webserver.cc:460] Webserver started at http://127.22.2.131:43027/ using document root <none> and password file <none>
I20260808 07:15:13.180209 12023 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:13.180249 12023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:13.180339 12023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:13.181325 12023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "b7f8b864dd9e49aaa76a8b5557795a00"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "a7910cbcc05a69aeae70cf3d9276de69"
server_key_iv: "a6ad784dfad1b763d617e4df8e4c4553"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.181628 12023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "b7f8b864dd9e49aaa76a8b5557795a00"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "a7910cbcc05a69aeae70cf3d9276de69"
server_key_iv: "a6ad784dfad1b763d617e4df8e4c4553"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.182894 12023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:15:13.183590 12041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.183764 12023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.183900 12023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "b7f8b864dd9e49aaa76a8b5557795a00"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "a7910cbcc05a69aeae70cf3d9276de69"
server_key_iv: "a6ad784dfad1b763d617e4df8e4c4553"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.183974 12023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:13.196708 12023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:13.198601 12023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:13.198755 12023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:13.199007 12023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:13.199327 12023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:13.199381 12023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.199424 12023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:13.199455 12023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.205509 12023 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:36289
I20260808 07:15:13.205555 12169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:36289 every 8 connection(s)
I20260808 07:15:13.205916 12023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:15:13 dist-test-slave-6k22 krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173313, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.131@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:13.214406 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 12023
I20260808 07:15:13.214529 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:15:13.214715 22538 external_mini_cluster.cc:1428] Setting key 8dbb2696ea704384845ae517b85cf443
I20260808 07:15:13.214955 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.206630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:56143 (local address 127.22.2.190:42003)
0808 07:15:13.206696 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.206698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.206934 (+ 236us) server_negotiation.cc:407] Connection header received
0808 07:15:13.207427 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.207429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.207440 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.207483 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:13.208467 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.208968 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.209909 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.210094 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.210144 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.212353 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:13.212355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:13.212355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:13.212362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:13.213543 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:13.213944 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:13.213945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:13.213946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:13.214004 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:13.214253 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:13.214254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:13.214254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:13.214377 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:13.214449 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.214770 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.214860 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:15:13.215170 12173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.206612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42003 (local address 127.22.2.131:56143)
0808 07:15:13.206774 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.206830 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:13.207329 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:13.207583 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:13.207588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:13.208192 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:13.208389 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.208395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.209044 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.209045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.209841 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.209843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.210162 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.210162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.210259 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.210761 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:13.210769 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:13.212248 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:13.213629 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:13.213637 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:13.213639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:13.213867 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:13.214071 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:13.214072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:13.214073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:13.214180 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:13.214441 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:13.214448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:13.214625 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 07:15:13.214747 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 07:15:13.214920 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260808 07:15:13.215788 12171 heartbeater.cc:347] Connected to a master server at 127.22.2.190:42003
I20260808 07:15:13.215902 12171 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:13.216079 12171 heartbeater.cc:511] Master 127.22.2.190:42003 requested a full tablet report, sending...
I20260808 07:15:13.216478 11662 ts_manager.cc:194] Registered new tserver with Master: b7f8b864dd9e49aaa76a8b5557795a00 (127.22.2.131:36289)
I20260808 07:15:13.216866 11662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.2.131@KRBTEST.COM'} at 127.22.2.131:56143
I20260808 07:15:13.226871 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:15:13.234086 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.228217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.22.2.190:42003)
0808 07:15:13.228282 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.228285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.228368 (+ 83us) server_negotiation.cc:407] Connection header received
0808 07:15:13.228440 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.228441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.228455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.228516 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:13.229084 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.229581 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.230329 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.230517 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.230590 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.231783 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:13.231785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:13.231785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:13.231793 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:13.232894 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:13.233260 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:13.233262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:13.233262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:13.233331 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:13.233524 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:13.233525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:13.233525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:13.233675 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:13.233784 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.233896 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.233981 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:15:13.235746 11662 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:15:13.237286 11662 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:15:13.243824 12185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.239588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:36289 (local address 127.0.0.1:49384)
0808 07:15:13.240095 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.240105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:13.240165 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:13.240494 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:13.240495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:13.240497 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:13.240548 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.240549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.241408 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.241409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.242603 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.242606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.243461 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.243462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.243542 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.243589 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:15:13.243666 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:15:13.243731 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":118,"threads_started":1}
I20260808 07:15:13.243834 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.239866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49384 (local address 127.22.2.131:36289)
0808 07:15:13.240043 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.240048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.240228 (+ 180us) server_negotiation.cc:407] Connection header received
0808 07:15:13.240308 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.240310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.240327 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.240411 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:13.240609 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.241303 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.242682 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.243372 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.243451 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.243511 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.243681 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.243738 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260808 07:15:13.244118 12183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.239584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:38567 (local address 127.0.0.1:48326)
0808 07:15:13.239752 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.239789 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:13.239872 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:13.240365 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:13.240368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:13.240370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:13.240439 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.240441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.241804 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.241805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.242826 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.242828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.243748 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.243749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.243863 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.243870 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:15:13.243907 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 07:15:13.244044 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260808 07:15:13.244481 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.239997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48326 (local address 127.22.2.130:38567)
0808 07:15:13.240177 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.240181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.240191 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:13.240236 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.240238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.240255 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.240521 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:13.240596 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.241694 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.242904 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.243647 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.244234 (+ 587us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.244317 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.244344 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.244387 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260808 07:15:13.244776 12096 tablet_service.cc:1467] Processing CreateTablet for tablet 7b403c7827b243e2b34e063e9dd3f76e (DEFAULT_TABLE table=test-table [id=666f14cbf00c4692810b6ebfae881dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:13.245062 11948 tablet_service.cc:1467] Processing CreateTablet for tablet 7b403c7827b243e2b34e063e9dd3f76e (DEFAULT_TABLE table=test-table [id=666f14cbf00c4692810b6ebfae881dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:13.245170 12096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b403c7827b243e2b34e063e9dd3f76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:13.245231 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.239584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:32963 (local address 127.0.0.1:44796)
0808 07:15:13.240056 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.240066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:13.240118 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:13.240419 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:13.240420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:13.240422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:13.240459 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.240460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.242902 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.242902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.243669 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.243672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.244957 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.244958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.245055 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.245060 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:15:13.245123 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:15:13.245163 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260808 07:15:13.245293 12186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.239730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.22.2.129:32963)
0808 07:15:13.240033 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.240040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.240185 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:15:13.240236 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.240238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.240255 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.240342 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:13.240537 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.241694 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.243757 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.244859 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.244990 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.245054 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.245137 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.245175 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":80,"threads_started":1}
I20260808 07:15:13.245393 11948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b403c7827b243e2b34e063e9dd3f76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:13.246037 11803 tablet_service.cc:1467] Processing CreateTablet for tablet 7b403c7827b243e2b34e063e9dd3f76e (DEFAULT_TABLE table=test-table [id=666f14cbf00c4692810b6ebfae881dbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:13.246340 11803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b403c7827b243e2b34e063e9dd3f76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:13.248862 12189 tablet_bootstrap.cc:492] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: Bootstrap starting.
I20260808 07:15:13.249408 12191 tablet_bootstrap.cc:492] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f: Bootstrap starting.
I20260808 07:15:13.249733 12189 tablet_bootstrap.cc:654] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:13.250000 12191 tablet_bootstrap.cc:654] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:13.250272 12189 log.cc:824] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:13.250538 12191 log.cc:824] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:13.250735 12190 tablet_bootstrap.cc:492] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Bootstrap starting.
I20260808 07:15:13.251173 12189 tablet_bootstrap.cc:492] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: No bootstrap required, opened a new log
I20260808 07:15:13.251228 12189 ts_tablet_manager.cc:1397] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:15:13.251273 12191 tablet_bootstrap.cc:492] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f: No bootstrap required, opened a new log
I20260808 07:15:13.251361 12191 ts_tablet_manager.cc:1397] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:15:13.252694 12189 raft_consensus.cc:348] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.252904 12189 raft_consensus.cc:374] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:13.252802 12191 raft_consensus.cc:348] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.252971 12189 raft_consensus.cc:729] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39275a9b80bc4fd7830334b7eca3dab7, State: Initialized, Role: FOLLOWER
I20260808 07:15:13.253014 12191 raft_consensus.cc:374] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:13.253057 12191 raft_consensus.cc:729] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2361b847ff7b41d0ab44ddecfb39b28f, State: Initialized, Role: FOLLOWER
I20260808 07:15:13.253093 12189 consensus_queue.cc:260] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.253173 12191 consensus_queue.cc:260] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.253409 12017 heartbeater.cc:503] Master 127.22.2.190:42003 was elected leader, sending a full tablet report...
I20260808 07:15:13.253494 12191 ts_tablet_manager.cc:1428] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:15:13.253505 11874 heartbeater.cc:503] Master 127.22.2.190:42003 was elected leader, sending a full tablet report...
I20260808 07:15:13.253410 12189 ts_tablet_manager.cc:1428] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:15:13.254087 12190 tablet_bootstrap.cc:654] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:13.254699 12190 log.cc:824] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:13.255465 12190 tablet_bootstrap.cc:492] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: No bootstrap required, opened a new log
I20260808 07:15:13.255523 12190 ts_tablet_manager.cc:1397] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 07:15:13.256898 12190 raft_consensus.cc:348] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.257035 12190 raft_consensus.cc:374] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:13.257057 12190 raft_consensus.cc:729] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f8b864dd9e49aaa76a8b5557795a00, State: Initialized, Role: FOLLOWER
I20260808 07:15:13.257138 12190 consensus_queue.cc:260] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.257380 12190 ts_tablet_manager.cc:1428] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:15:13.257416 12171 heartbeater.cc:503] Master 127.22.2.190:42003 was elected leader, sending a full tablet report...
W20260808 07:15:13.293504 12018 tablet.cc:2369] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:15:13.372133 11875 tablet.cc:2369] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:15:13.456576 12172 tablet.cc:2369] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:15:13.479080 12194 raft_consensus.cc:482] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:15:13.479234 12194 raft_consensus.cc:504] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.479733 12194 leader_election.cc:288] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2361b847ff7b41d0ab44ddecfb39b28f (127.22.2.129:32963), b7f8b864dd9e49aaa76a8b5557795a00 (127.22.2.131:36289)
I20260808 07:15:13.482659 12186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.480047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:59911 (local address 127.22.2.129:32963)
0808 07:15:13.480097 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.480100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.480113 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:15:13.480146 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.480148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.480160 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.480246 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:13.480444 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.480976 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.481947 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.482305 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.482361 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.482368 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.482548 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.482587 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:15:13.482672 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.480055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:57089 (local address 127.22.2.131:36289)
0808 07:15:13.480103 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.480105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.480280 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:15:13.480316 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.480317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.480328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.480429 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:13.480570 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.481030 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.481944 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.482319 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.482412 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.482419 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.482571 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.482610 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:15:13.482667 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.479953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:32963 (local address 127.22.2.130:59911)
0808 07:15:13.480015 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.480027 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:13.480075 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:13.480278 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:13.480279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:13.480281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:13.480364 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.480366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.481058 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.481059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.481849 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.481851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.482387 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.482389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.482476 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.482479 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:13.482521 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 07:15:13.482582 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 07:15:13.482681 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.479953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:36289 (local address 127.22.2.130:57089)
0808 07:15:13.480196 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.480207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:13.480268 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:13.480434 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:13.480434 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:13.480436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:13.480503 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.480505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.481109 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.481110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.481851 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.481852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.482422 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.482423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.482482 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.482485 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:13.482552 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:15:13.482589 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":23,"thread_start_us":132,"threads_started":1}
I20260808 07:15:13.483053 11824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b403c7827b243e2b34e063e9dd3f76e" candidate_uuid: "39275a9b80bc4fd7830334b7eca3dab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" is_pre_election: true
I20260808 07:15:13.483211 11824 raft_consensus.cc:2454] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39275a9b80bc4fd7830334b7eca3dab7 in term 0.
I20260808 07:15:13.483354 12118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b403c7827b243e2b34e063e9dd3f76e" candidate_uuid: "39275a9b80bc4fd7830334b7eca3dab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" is_pre_election: true
I20260808 07:15:13.483453 11897 leader_election.cc:302] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2361b847ff7b41d0ab44ddecfb39b28f, 39275a9b80bc4fd7830334b7eca3dab7; no voters:
I20260808 07:15:13.483516 12118 raft_consensus.cc:2454] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39275a9b80bc4fd7830334b7eca3dab7 in term 0.
I20260808 07:15:13.483598 12194 raft_consensus.cc:2781] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:15:13.483669 12194 raft_consensus.cc:482] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:15:13.483698 12194 raft_consensus.cc:3037] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:13.484439 12194 raft_consensus.cc:504] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.484580 12194 leader_election.cc:288] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [CANDIDATE]: Term 1 election: Requested vote from peers 2361b847ff7b41d0ab44ddecfb39b28f (127.22.2.129:32963), b7f8b864dd9e49aaa76a8b5557795a00 (127.22.2.131:36289)
I20260808 07:15:13.484745 11824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b403c7827b243e2b34e063e9dd3f76e" candidate_uuid: "39275a9b80bc4fd7830334b7eca3dab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2361b847ff7b41d0ab44ddecfb39b28f"
I20260808 07:15:13.484743 12118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b403c7827b243e2b34e063e9dd3f76e" candidate_uuid: "39275a9b80bc4fd7830334b7eca3dab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f8b864dd9e49aaa76a8b5557795a00"
I20260808 07:15:13.484824 11824 raft_consensus.cc:3037] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:13.484824 12118 raft_consensus.cc:3037] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:13.485726 12118 raft_consensus.cc:2454] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39275a9b80bc4fd7830334b7eca3dab7 in term 1.
I20260808 07:15:13.485726 11824 raft_consensus.cc:2454] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39275a9b80bc4fd7830334b7eca3dab7 in term 1.
I20260808 07:15:13.485908 11899 leader_election.cc:302] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39275a9b80bc4fd7830334b7eca3dab7, b7f8b864dd9e49aaa76a8b5557795a00; no voters:
I20260808 07:15:13.486003 12194 raft_consensus.cc:2781] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:13.486150 12194 raft_consensus.cc:686] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 1 LEADER]: Becoming Leader. State: Replica: 39275a9b80bc4fd7830334b7eca3dab7, State: Running, Role: LEADER
I20260808 07:15:13.486251 12194 consensus_queue.cc:237] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } }
I20260808 07:15:13.487133 11662 catalog_manager.cc:5495] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39275a9b80bc4fd7830334b7eca3dab7 (127.22.2.130). New cstate: current_term: 1 leader_uuid: "39275a9b80bc4fd7830334b7eca3dab7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39275a9b80bc4fd7830334b7eca3dab7" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 38567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 } health_report { overall_health: UNKNOWN } } }
I20260808 07:15:13.526294 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.523770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.22.2.130:38567)
0808 07:15:13.523841 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.523843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.523856 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:15:13.523960 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.523961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.523984 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.524065 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:13.524217 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.524722 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.525481 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.525690 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.525766 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.525930 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:13.526040 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:13.526125 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.526149 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.526169 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:15:13.528237 11824 raft_consensus.cc:1261] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 1 FOLLOWER]: Refusing update from remote peer 39275a9b80bc4fd7830334b7eca3dab7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:13.528237 12118 raft_consensus.cc:1261] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 1 FOLLOWER]: Refusing update from remote peer 39275a9b80bc4fd7830334b7eca3dab7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:13.528654 12194 consensus_queue.cc:1035] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7f8b864dd9e49aaa76a8b5557795a00" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 36289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:13.528817 12199 consensus_queue.cc:1035] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2361b847ff7b41d0ab44ddecfb39b28f" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:13.531319 12203 mvcc.cc:205] Tried to move back new op lower bound from 7316165892208562176 to 7316165892040302592. Current Snapshot: MvccSnapshot[applied={T|T < 7316165892208562176}]
I20260808 07:15:13.531944 12204 mvcc.cc:205] Tried to move back new op lower bound from 7316165892208562176 to 7316165892040302592. Current Snapshot: MvccSnapshot[applied={T|T < 7316165892208562176}]
I20260808 07:15:13.533826 11662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:13.533933 11662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:13.534715 11662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 377f8cd9773a45ff81cb383fa9896f84: Sending DeleteTablet for 3 replicas of tablet 7b403c7827b243e2b34e063e9dd3f76e
I20260808 07:15:13.535059 11948 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b403c7827b243e2b34e063e9dd3f76e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:13 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:48326
I20260808 07:15:13.535122 12096 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b403c7827b243e2b34e063e9dd3f76e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:13 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:49384
I20260808 07:15:13.535122 11803 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b403c7827b243e2b34e063e9dd3f76e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:13 UTC) from {username='kudu', principal='kudu/127.22.2.190@KRBTEST.COM'} at 127.0.0.1:44796
I20260808 07:15:13.535365 12215 tablet_replica.cc:330] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: stopping tablet replica
I20260808 07:15:13.535445 12216 tablet_replica.cc:330] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f: stopping tablet replica
I20260808 07:15:13.535586 12216 raft_consensus.cc:2229] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:13.535634 12215 raft_consensus.cc:2229] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:13.535764 12215 raft_consensus.cc:2258] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:13.535772 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 11732
I20260808 07:15:13.535794 12216 raft_consensus.cc:2258] T 7b403c7827b243e2b34e063e9dd3f76e P 2361b847ff7b41d0ab44ddecfb39b28f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:13.542338 12215 ts_tablet_manager.cc:1905] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:13.544795 12215 ts_tablet_manager.cc:1918] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:13.544955 12215 log.cc:1196] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/7b403c7827b243e2b34e063e9dd3f76e
I20260808 07:15:13.545372 12215 ts_tablet_manager.cc:1939] T 7b403c7827b243e2b34e063e9dd3f76e P b7f8b864dd9e49aaa76a8b5557795a00: Deleting consensus metadata
I20260808 07:15:13.545881 11648 catalog_manager.cc:4841] TS b7f8b864dd9e49aaa76a8b5557795a00 (127.22.2.131:36289): tablet 7b403c7827b243e2b34e063e9dd3f76e (table test-table [id=666f14cbf00c4692810b6ebfae881dbd]) successfully deleted
I20260808 07:15:13.547737 12214 tablet_replica.cc:330] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: stopping tablet replica
I20260808 07:15:13.547901 12214 raft_consensus.cc:2229] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:15:13.548038 12214 raft_consensus.cc:2258] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:13.548471 12214 ts_tablet_manager.cc:1905] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:15:13.549993 11646 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:32963 (error 108)
I20260808 07:15:13.550216 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 11879
I20260808 07:15:13.550232 12214 ts_tablet_manager.cc:1918] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:13.550307 12214 log.cc:1196] T 7b403c7827b243e2b34e063e9dd3f76e P 39275a9b80bc4fd7830334b7eca3dab7: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/7b403c7827b243e2b34e063e9dd3f76e
I20260808 07:15:13.553646 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.553499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:32963 (local address 127.0.0.1:44810)
0808 07:15:13.553552 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.553596 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:32963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 07:15:13.553797 11646 catalog_manager.cc:4584] TS 2361b847ff7b41d0ab44ddecfb39b28f (127.22.2.129:32963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b403c7827b243e2b34e063e9dd3f76e: Network error: Client connection negotiation failed: client connection to 127.22.2.129:32963: connect: Connection refused (error 111)
I20260808 07:15:13.557646 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.557510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:38567 (local address 127.0.0.1:48338)
0808 07:15:13.557553 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.557601 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.130:38567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:15:13.557670 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 12023
I20260808 07:15:13.563934 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 11628
2026-08-08T07:15:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2286 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T07:15:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:13Z Disabled control of system clock
I20260808 07:15:13.595608 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33681
--webserver_interface=127.22.2.190
--webserver_port=0
--builtin_ntp_servers=127.22.2.148:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:33681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:15:13.678402 12222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:13.678624 12222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:13.678644 12222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:13.679816 12222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:13.679854 12222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:13.679867 12222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:13.679880 12222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:13.679891 12222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:13.681166 12222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:33681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:33681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:13.681373 12222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:13.681617 12222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:15:13.686457 12222 server_base.cc:1034] running on GCE node
W20260808 07:15:13.686437 12229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:13.686440 12231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:13.686450 12227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:13.686877 12222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:13.687160 12222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:13.688294 12222 hybrid_clock.cc:648] HybridClock initialized: now 1786173313688278 us; error 32 us; skew 500 ppm
I20260808 07:15:13.689545 12222 webserver.cc:460] Webserver started at http://127.22.2.190:36855/ using document root <none> and password file <none>
I20260808 07:15:13.689811 12222 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:13.689864 12222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:13.690018 12222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:13.691098 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "ea770ed8c3d949f9ba96ee96593ca78b"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "a351d463d8da1e88ff4dc4ba8304ac3c"
server_key_iv: "9bd25196349560790405fcb787db869a"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.691473 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "ea770ed8c3d949f9ba96ee96593ca78b"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "a351d463d8da1e88ff4dc4ba8304ac3c"
server_key_iv: "9bd25196349560790405fcb787db869a"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.692808 12222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:15:13.693553 12238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.693715 12222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:13.693813 12222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "ea770ed8c3d949f9ba96ee96593ca78b"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "a351d463d8da1e88ff4dc4ba8304ac3c"
server_key_iv: "9bd25196349560790405fcb787db869a"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.693871 12222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:13.705085 12222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:13.705363 12222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:13.705467 12222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:13.709373 12222 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:33681
I20260808 07:15:13.709438 12293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:33681 every 8 connection(s)
I20260808 07:15:13.709793 12222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:13.710134 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 12222
I20260808 07:15:13.710261 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:13.710484 22538 external_mini_cluster.cc:1428] Setting key 897bfe49f2f034a2d567ee90a92e8616
I20260808 07:15:13.711042 12294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:13.714303 12294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b: Bootstrap starting.
I20260808 07:15:13.714999 12294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:13.715337 12294 log.cc:824] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:13.716183 12294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b: No bootstrap required, opened a new log
I20260808 07:15:13.716789 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.711036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.22.2.190:33681)
0808 07:15:13.711258 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.711268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.711297 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:15:13.711861 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.711870 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.711962 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.712113 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:13.712307 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.713629 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.714380 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.715004 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.715141 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.716089 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:13.716094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:13.716094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:13.716307 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:13.716354 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:13.716369 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:13.716439 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.716498 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.716574 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260808 07:15:13.718286 12294 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea770ed8c3d949f9ba96ee96593ca78b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33681 } }
I20260808 07:15:13.718434 12294 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:13.718458 12294 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea770ed8c3d949f9ba96ee96593ca78b, State: Initialized, Role: FOLLOWER
I20260808 07:15:13.718539 12294 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea770ed8c3d949f9ba96ee96593ca78b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33681 } }
I20260808 07:15:13.718588 12294 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:13.718609 12294 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:13.718636 12294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:13.719121 12294 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea770ed8c3d949f9ba96ee96593ca78b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33681 } }
I20260808 07:15:13.719203 12294 leader_election.cc:302] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea770ed8c3d949f9ba96ee96593ca78b; no voters:
I20260808 07:15:13.719349 12294 leader_election.cc:288] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:13.719409 12300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:13.719609 12300 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [term 1 LEADER]: Becoming Leader. State: Replica: ea770ed8c3d949f9ba96ee96593ca78b, State: Running, Role: LEADER
I20260808 07:15:13.719647 12294 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:13.719702 12300 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea770ed8c3d949f9ba96ee96593ca78b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33681 } }
I20260808 07:15:13.720070 12302 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea770ed8c3d949f9ba96ee96593ca78b. Latest consensus state: current_term: 1 leader_uuid: "ea770ed8c3d949f9ba96ee96593ca78b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea770ed8c3d949f9ba96ee96593ca78b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33681 } } }
I20260808 07:15:13.720063 12301 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea770ed8c3d949f9ba96ee96593ca78b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea770ed8c3d949f9ba96ee96593ca78b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 33681 } } }
I20260808 07:15:13.720166 12302 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:13.720175 12301 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:13.720400 12308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:13.720916 12308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:13.724560 12308 catalog_manager.cc:1347] Generated new cluster ID: f96749338aa14962b203b941d2962da8
I20260808 07:15:13.724614 12308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:13.733392 12308 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:13.734087 12308 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:13.742085 12308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b: Generated new TSK 0
I20260808 07:15:13.742326 12308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:13.744134 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.129:0
--local_ip_for_outbound_sockets=127.22.2.129
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33681
--builtin_ntp_servers=127.22.2.148:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:15:13.827816 12320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:13.828053 12320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:13.828091 12320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:13.829296 12320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:13.829350 12320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:13.829387 12320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.129
I20260808 07:15:13.830706 12320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.2.129
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.129
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:13.830964 12320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:13.831252 12320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:13.837188 12328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:13.837256 12326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:13.837201 12325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:13.837494 12320 server_base.cc:1034] running on GCE node
I20260808 07:15:13.837702 12320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:13.837962 12320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:13.839116 12320 hybrid_clock.cc:648] HybridClock initialized: now 1786173313839104 us; error 32 us; skew 500 ppm
I20260808 07:15:13.840459 12320 webserver.cc:460] Webserver started at http://127.22.2.129:46405/ using document root <none> and password file <none>
I20260808 07:15:13.840700 12320 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:13.840770 12320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:13.840914 12320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:13.841975 12320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/data/instance:
uuid: "90f427b29c2843a881082211e13781c3"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "0c73c998f32d9ceff352e07cc936426c"
server_key_iv: "cadafa1c9fe2fe63c032671f9786a404"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.842344 12320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance:
uuid: "90f427b29c2843a881082211e13781c3"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "0c73c998f32d9ceff352e07cc936426c"
server_key_iv: "cadafa1c9fe2fe63c032671f9786a404"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.843794 12320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:13.844539 12334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.844726 12320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:13.844823 12320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/wal
uuid: "90f427b29c2843a881082211e13781c3"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "0c73c998f32d9ceff352e07cc936426c"
server_key_iv: "cadafa1c9fe2fe63c032671f9786a404"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.844898 12320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:13.865401 12320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:13.865746 12320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:13.865901 12320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:13.866156 12320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:13.866509 12320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:13.866561 12320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.866603 12320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:13.866626 12320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.872462 12320 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.129:37979
I20260808 07:15:13.872567 12460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.129:37979 every 8 connection(s)
I20260808 07:15:13.872860 12320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/data/info.pb
I20260808 07:15:13.877967 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.873640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.129:34459 (local address 127.22.2.190:33681)
0808 07:15:13.873706 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:13.873707 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:13.873935 (+ 228us) server_negotiation.cc:407] Connection header received
0808 07:15:13.874426 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:13.874427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:13.874441 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:13.874498 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:13.875177 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.875722 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.876650 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.876835 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.876884 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.877636 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:13.877637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:13.877638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:13.877642 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:13.877656 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:13.877660 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:13.877703 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:13.877865 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:15:13.877902 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:15:13.878269 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:13.873619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33681 (local address 127.22.2.129:34459)
0808 07:15:13.873823 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:13.873859 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:13.874340 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:13.874571 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:13.874575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:13.874583 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:13.875102 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.875106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.875794 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.875795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.876578 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:13.876580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:13.876904 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:13.876904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:13.877013 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:13.877510 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:13.877510 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:13.877539 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:13.877561 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:13.877769 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:13.877778 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:13.877780 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:15:13.877862 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:15:13.877964 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260808 07:15:13.878880 12461 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33681
I20260808 07:15:13.878963 12461 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:13.879117 12461 heartbeater.cc:511] Master 127.22.2.190:33681 requested a full tablet report, sending...
I20260808 07:15:13.879691 12257 ts_manager.cc:194] Registered new tserver with Master: 90f427b29c2843a881082211e13781c3 (127.22.2.129:37979)
I20260808 07:15:13.879954 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 12320
I20260808 07:15:13.880054 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-0/wal/instance
I20260808 07:15:13.880252 22538 external_mini_cluster.cc:1428] Setting key 2659e3b2d907b6c5d978ca56e31c6846
I20260808 07:15:13.880527 12257 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.129:34459
I20260808 07:15:13.881345 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.130:0
--local_ip_for_outbound_sockets=127.22.2.130
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33681
--builtin_ntp_servers=127.22.2.148:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:15:13.966617 12464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:13.966861 12464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:13.966885 12464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:13.968180 12464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:13.968225 12464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:13.968251 12464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.130
I20260808 07:15:13.970090 12464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.2.130
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.130
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:13.970367 12464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:13.970691 12464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:15:13.976305 12464 server_base.cc:1034] running on GCE node
W20260808 07:15:13.976192 12471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:13.976192 12470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:13.976245 12473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:13.976761 12464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:13.977017 12464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:13.978174 12464 hybrid_clock.cc:648] HybridClock initialized: now 1786173313978181 us; error 51 us; skew 500 ppm
I20260808 07:15:13.979493 12464 webserver.cc:460] Webserver started at http://127.22.2.130:43995/ using document root <none> and password file <none>
I20260808 07:15:13.979756 12464 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:13.979837 12464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:13.979992 12464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:13.981076 12464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/data/instance:
uuid: "7d8fece9efae4563948d190e71074028"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "fa378c62085221a2fd4105b34a77590b"
server_key_iv: "b06955e68b157b40e1dfb569616ef299"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.981446 12464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance:
uuid: "7d8fece9efae4563948d190e71074028"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "fa378c62085221a2fd4105b34a77590b"
server_key_iv: "b06955e68b157b40e1dfb569616ef299"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.982730 12464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:13.983429 12479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:13.983632 12464 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:13.983748 12464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/wal
uuid: "7d8fece9efae4563948d190e71074028"
format_stamp: "Formatted at 2026-08-08 07:15:13 on dist-test-slave-6k22"
server_key: "fa378c62085221a2fd4105b34a77590b"
server_key_iv: "b06955e68b157b40e1dfb569616ef299"
server_key_version: "encryptionkey@0"
I20260808 07:15:13.983831 12464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:14.006044 12464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:14.006394 12464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:14.006541 12464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:14.006783 12464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:14.007120 12464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:14.007172 12464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:14.007213 12464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:14.007251 12464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:14.013712 12464 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.130:44405
I20260808 07:15:14.013756 12606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.130:44405 every 8 connection(s)
I20260808 07:15:14.014082 12464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/data/info.pb
I20260808 07:15:14.017192 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 12464
I20260808 07:15:14.017298 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/wal/instance
I20260808 07:15:14.017529 22538 external_mini_cluster.cc:1428] Setting key d01da64822780b88d76b2f99605d7321
I20260808 07:15:14.018498 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.2.131:0
--local_ip_for_outbound_sockets=127.22.2.131
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33681
--builtin_ntp_servers=127.22.2.148:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:15:14.019506 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.014728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.130:55441 (local address 127.22.2.190:33681)
0808 07:15:14.014787 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.014789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.015029 (+ 240us) server_negotiation.cc:407] Connection header received
0808 07:15:14.015503 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.015504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.015513 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.015641 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.016309 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.016793 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.017851 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.018053 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.018130 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.019115 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.019117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.019117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:14.019123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.019140 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:14.019145 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.019221 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.019360 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.019388 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:15:14.019639 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.014726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33681 (local address 127.22.2.130:55441)
0808 07:15:14.014902 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.014943 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.015424 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.015714 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.015717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.015725 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.016232 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.016236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.016868 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.016868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.017778 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.017781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.018168 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.018170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.018326 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.018982 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:14.018983 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.019010 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:14.019032 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.019266 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.019273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.019276 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.019342 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.019423 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260808 07:15:14.020144 12607 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33681
I20260808 07:15:14.020247 12607 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:14.020519 12607 heartbeater.cc:511] Master 127.22.2.190:33681 requested a full tablet report, sending...
I20260808 07:15:14.020953 12256 ts_manager.cc:194] Registered new tserver with Master: 7d8fece9efae4563948d190e71074028 (127.22.2.130:44405)
I20260808 07:15:14.021327 12256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.130:55441
W20260808 07:15:14.102236 12610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:14.102451 12610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:14.102473 12610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:14.103541 12610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:14.103641 12610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:14.103670 12610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.2.131
I20260808 07:15:14.104997 12610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.2.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.2.131
--webserver_port=0
--tserver_master_addrs=127.22.2.190:33681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.2.131
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:14.105178 12610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:14.105425 12610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:14.110487 12615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:14.110463 12618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:14.110476 12616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:14.110653 12610 server_base.cc:1034] running on GCE node
I20260808 07:15:14.110926 12610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:14.111157 12610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:14.112306 12610 hybrid_clock.cc:648] HybridClock initialized: now 1786173314112301 us; error 34 us; skew 500 ppm
I20260808 07:15:14.113547 12610 webserver.cc:460] Webserver started at http://127.22.2.131:42527/ using document root <none> and password file <none>
I20260808 07:15:14.113783 12610 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:14.113862 12610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:14.114014 12610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:14.115078 12610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/data/instance:
uuid: "c8d45d28b3e74bffbee8034f07176e4e"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-6k22"
server_key: "3f9096564b049076de99e3b922a43d8b"
server_key_iv: "2d0008f433138f43eb15193ad41f93ca"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.115440 12610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance:
uuid: "c8d45d28b3e74bffbee8034f07176e4e"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-6k22"
server_key: "3f9096564b049076de99e3b922a43d8b"
server_key_iv: "2d0008f433138f43eb15193ad41f93ca"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.116760 12610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:15:14.117457 12624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:14.117625 12610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:14.117722 12610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/wal
uuid: "c8d45d28b3e74bffbee8034f07176e4e"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-6k22"
server_key: "3f9096564b049076de99e3b922a43d8b"
server_key_iv: "2d0008f433138f43eb15193ad41f93ca"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.117803 12610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:14.138206 12610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:14.138547 12610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:14.138695 12610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:14.138928 12610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:15:14.139268 12610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:15:14.139320 12610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:14.139353 12610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:15:14.139379 12610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:14.145229 12610 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.131:34405
I20260808 07:15:14.145273 12755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.131:34405 every 8 connection(s)
I20260808 07:15:14.145603 12610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/data/info.pb
I20260808 07:15:14.150938 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.146381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:35865 (local address 127.22.2.190:33681)
0808 07:15:14.146428 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.146431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.146637 (+ 206us) server_negotiation.cc:407] Connection header received
0808 07:15:14.147168 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.147170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.147184 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.147237 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.148033 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.148544 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.149593 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.149795 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.149872 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.150646 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.150648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.150648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:14.150653 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.150667 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:14.150671 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.150735 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.150832 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.150858 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:14.151167 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.146346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:33681 (local address 127.22.2.131:35865)
0808 07:15:14.146518 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.146553 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.147065 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.147312 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.147316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.147330 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.147946 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.147953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.148648 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.148649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.149505 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.149507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.149897 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.149898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.150021 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.150510 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:15:14.150510 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.150538 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:15:14.150559 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.150739 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.150744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.150747 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.150827 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.150928 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260808 07:15:14.151655 12756 heartbeater.cc:347] Connected to a master server at 127.22.2.190:33681
I20260808 07:15:14.151759 12756 heartbeater.cc:464] Registering TS with master...
I20260808 07:15:14.151952 12756 heartbeater.cc:511] Master 127.22.2.190:33681 requested a full tablet report, sending...
I20260808 07:15:14.152307 12256 ts_manager.cc:194] Registered new tserver with Master: c8d45d28b3e74bffbee8034f07176e4e (127.22.2.131:34405)
I20260808 07:15:14.152679 12256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.2.131:35865
I20260808 07:15:14.154567 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 12610
I20260808 07:15:14.154693 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/wal/instance
I20260808 07:15:14.154899 22538 external_mini_cluster.cc:1428] Setting key 15babc7c612eba5cf4b3c993088e17a1
I20260808 07:15:14.155689 22538 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:15:14.160018 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.157080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.22.2.190:33681)
0808 07:15:14.157143 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.157145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.157278 (+ 133us) server_negotiation.cc:407] Connection header received
0808 07:15:14.157374 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.157375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.157386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.157443 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.157603 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.158080 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.158850 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.159053 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.159126 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.159764 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.159766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.159766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:14.159771 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.159785 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:14.159789 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.159850 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.159910 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.159932 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:15:14.161427 12256 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:15:14.162935 12256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:15:14.169251 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.165112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44405 (local address 127.0.0.1:36102)
0808 07:15:14.165404 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.165415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.165456 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.165750 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.165754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.165756 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:14.165825 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.165827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.167165 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.167167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.168293 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.168295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.168951 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.168952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.169027 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.169039 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.169110 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.169163 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":90,"threads_started":1}
I20260808 07:15:14.169361 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.165176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.22.2.131:34405)
0808 07:15:14.165845 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.165850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.165861 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:15:14.165895 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.165897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.165911 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.166117 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:14.166175 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.166825 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.168309 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.169000 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.169094 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.169170 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.169226 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.169275 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":87,"threads_started":1}
I20260808 07:15:14.169402 12766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.165090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.131:34405 (local address 127.0.0.1:40550)
0808 07:15:14.165287 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.165339 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.165407 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.166005 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.166006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.166007 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:14.166045 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.166047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.166908 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.166909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.168231 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.168234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.169104 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.169105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.169157 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.169160 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.169216 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.169247 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":60,"thread_start_us":80,"threads_started":1}
I20260808 07:15:14.169432 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.165224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36102 (local address 127.22.2.130:44405)
0808 07:15:14.165555 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.165560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.165570 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:14.165608 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.165609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.165621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.165714 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:14.165933 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.167079 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.168367 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.168854 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.168949 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.169043 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.169296 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.169336 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":83,"threads_started":1}
I20260808 07:15:14.169938 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.165555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.22.2.129:37979)
0808 07:15:14.165749 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.165753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.165763 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:14.165814 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.165816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.165830 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.165907 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:14.166099 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.167154 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.168229 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.168952 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.169028 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.169119 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.169786 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.169830 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
I20260808 07:15:14.170190 12769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.165317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37979 (local address 127.0.0.1:37170)
0808 07:15:14.165455 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.165467 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.165520 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.165979 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.165980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.165983 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:14.166029 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.166032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.167216 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.167217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.168146 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.168148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.169154 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.169155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.169222 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.169226 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.169278 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.169318 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260808 07:15:14.170118 12533 tablet_service.cc:1467] Processing CreateTablet for tablet b3b86da363684ad4b531e8384b484895 (DEFAULT_TABLE table=test-table [id=ee96dcc1601a45ab8cbf84be1f49371c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:14.170217 12677 tablet_service.cc:1467] Processing CreateTablet for tablet b3b86da363684ad4b531e8384b484895 (DEFAULT_TABLE table=test-table [id=ee96dcc1601a45ab8cbf84be1f49371c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:14.170511 12677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b86da363684ad4b531e8384b484895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:14.170511 12533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b86da363684ad4b531e8384b484895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:14.170879 12390 tablet_service.cc:1467] Processing CreateTablet for tablet b3b86da363684ad4b531e8384b484895 (DEFAULT_TABLE table=test-table [id=ee96dcc1601a45ab8cbf84be1f49371c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:15:14.171135 12390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b86da363684ad4b531e8384b484895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:14.174654 12772 tablet_bootstrap.cc:492] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3: Bootstrap starting.
I20260808 07:15:14.175449 12772 tablet_bootstrap.cc:654] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:14.175766 12771 tablet_bootstrap.cc:492] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: Bootstrap starting.
I20260808 07:15:14.175835 12772 log.cc:824] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:14.176190 12773 tablet_bootstrap.cc:492] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: Bootstrap starting.
I20260808 07:15:14.176462 12771 tablet_bootstrap.cc:654] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:14.176780 12771 log.cc:824] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:14.176843 12773 tablet_bootstrap.cc:654] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:14.177140 12773 log.cc:824] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:14.177747 12771 tablet_bootstrap.cc:492] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: No bootstrap required, opened a new log
I20260808 07:15:14.177742 12772 tablet_bootstrap.cc:492] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3: No bootstrap required, opened a new log
I20260808 07:15:14.177831 12771 ts_tablet_manager.cc:1397] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:15:14.177949 12772 ts_tablet_manager.cc:1397] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:15:14.180341 12771 raft_consensus.cc:348] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.180364 12772 raft_consensus.cc:348] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.180474 12771 raft_consensus.cc:374] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:14.180498 12771 raft_consensus.cc:729] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d8fece9efae4563948d190e71074028, State: Initialized, Role: FOLLOWER
I20260808 07:15:14.180500 12772 raft_consensus.cc:374] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:14.180537 12772 raft_consensus.cc:729] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f427b29c2843a881082211e13781c3, State: Initialized, Role: FOLLOWER
I20260808 07:15:14.180683 12772 consensus_queue.cc:260] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.180948 12773 tablet_bootstrap.cc:492] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: No bootstrap required, opened a new log
I20260808 07:15:14.180956 12772 ts_tablet_manager.cc:1428] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 07:15:14.181021 12773 ts_tablet_manager.cc:1397] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 07:15:14.181044 12461 heartbeater.cc:503] Master 127.22.2.190:33681 was elected leader, sending a full tablet report...
I20260808 07:15:14.183120 12773 raft_consensus.cc:348] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.183249 12773 raft_consensus.cc:374] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:14.183272 12773 raft_consensus.cc:729] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8d45d28b3e74bffbee8034f07176e4e, State: Initialized, Role: FOLLOWER
I20260808 07:15:14.183395 12773 consensus_queue.cc:260] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.183681 12773 ts_tablet_manager.cc:1428] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:15:14.183794 12756 heartbeater.cc:503] Master 127.22.2.190:33681 was elected leader, sending a full tablet report...
I20260808 07:15:14.183833 12771 consensus_queue.cc:260] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.184163 12771 ts_tablet_manager.cc:1428] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260808 07:15:14.184185 12607 heartbeater.cc:503] Master 127.22.2.190:33681 was elected leader, sending a full tablet report...
I20260808 07:15:14.187655 12779 raft_consensus.cc:482] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:15:14.187721 12779 raft_consensus.cc:504] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.188037 12779 leader_election.cc:288] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f427b29c2843a881082211e13781c3 (127.22.2.129:37979), 7d8fece9efae4563948d190e71074028 (127.22.2.130:44405)
I20260808 07:15:14.190843 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.188461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:46557 (local address 127.22.2.130:44405)
0808 07:15:14.188510 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.188512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.188520 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:14.188544 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.188545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.188552 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.188613 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:14.188768 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.189214 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.190148 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.190509 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.190587 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.190595 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.190741 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.190775 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:15:14.190851 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.188180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.130:44405 (local address 127.22.2.131:46557)
0808 07:15:14.188275 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.188288 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.188344 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.188617 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.188618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.188619 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:14.188687 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.188688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.189289 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.189290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.190042 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.190044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.190585 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.190586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.190654 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.190657 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.190733 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.190775 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":24}
I20260808 07:15:14.190920 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.188417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.2.131:53965 (local address 127.22.2.129:37979)
0808 07:15:14.188475 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.188477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.188485 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:14.188510 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.188511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.188519 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.188575 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:15:14.188738 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.189213 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.190190 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.190556 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.190654 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.190661 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.190817 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.190851 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:15:14.191048 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.188196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37979 (local address 127.22.2.131:53965)
0808 07:15:14.188342 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.188350 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.188392 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.188594 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.188595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.188597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:15:14.188664 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.188666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.189291 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.189292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.190062 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.190064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.190656 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.190657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.190736 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.190738 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.190808 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.190843 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260808 07:15:14.191205 12556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b86da363684ad4b531e8384b484895" candidate_uuid: "c8d45d28b3e74bffbee8034f07176e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d8fece9efae4563948d190e71074028" is_pre_election: true
I20260808 07:15:14.191341 12556 raft_consensus.cc:2454] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8d45d28b3e74bffbee8034f07176e4e in term 0.
I20260808 07:15:14.191637 12628 leader_election.cc:302] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d8fece9efae4563948d190e71074028, c8d45d28b3e74bffbee8034f07176e4e; no voters:
I20260808 07:15:14.191766 12779 raft_consensus.cc:2781] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:15:14.191735 12412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b86da363684ad4b531e8384b484895" candidate_uuid: "c8d45d28b3e74bffbee8034f07176e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f427b29c2843a881082211e13781c3" is_pre_election: true
I20260808 07:15:14.191818 12779 raft_consensus.cc:482] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:15:14.191835 12779 raft_consensus.cc:3037] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:14.191870 12412 raft_consensus.cc:2454] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8d45d28b3e74bffbee8034f07176e4e in term 0.
I20260808 07:15:14.192528 12779 raft_consensus.cc:504] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.192672 12779 leader_election.cc:288] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [CANDIDATE]: Term 1 election: Requested vote from peers 90f427b29c2843a881082211e13781c3 (127.22.2.129:37979), 7d8fece9efae4563948d190e71074028 (127.22.2.130:44405)
I20260808 07:15:14.192808 12412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b86da363684ad4b531e8384b484895" candidate_uuid: "c8d45d28b3e74bffbee8034f07176e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f427b29c2843a881082211e13781c3"
I20260808 07:15:14.192811 12556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b86da363684ad4b531e8384b484895" candidate_uuid: "c8d45d28b3e74bffbee8034f07176e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d8fece9efae4563948d190e71074028"
I20260808 07:15:14.192883 12412 raft_consensus.cc:3037] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:14.192883 12556 raft_consensus.cc:3037] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:14.193791 12556 raft_consensus.cc:2454] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8d45d28b3e74bffbee8034f07176e4e in term 1.
I20260808 07:15:14.193792 12412 raft_consensus.cc:2454] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8d45d28b3e74bffbee8034f07176e4e in term 1.
I20260808 07:15:14.193964 12628 leader_election.cc:302] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d8fece9efae4563948d190e71074028, c8d45d28b3e74bffbee8034f07176e4e; no voters:
I20260808 07:15:14.194072 12779 raft_consensus.cc:2781] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:14.194192 12779 raft_consensus.cc:686] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 1 LEADER]: Becoming Leader. State: Replica: c8d45d28b3e74bffbee8034f07176e4e, State: Running, Role: LEADER
I20260808 07:15:14.194291 12779 consensus_queue.cc:237] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } }
I20260808 07:15:14.195060 12257 catalog_manager.cc:5495] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to c8d45d28b3e74bffbee8034f07176e4e (127.22.2.131). New cstate: current_term: 1 leader_uuid: "c8d45d28b3e74bffbee8034f07176e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8d45d28b3e74bffbee8034f07176e4e" member_type: VOTER last_known_addr { host: "127.22.2.131" port: 34405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 } health_report { overall_health: UNKNOWN } } }
I20260808 07:15:14.206720 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.204153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40558 (local address 127.22.2.131:34405)
0808 07:15:14.204209 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.204211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.204222 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:15:14.204303 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.204304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.204324 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.204414 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:14.204557 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.205077 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.205905 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.206101 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.206188 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.206362 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:14.206459 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:14.206557 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.206579 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.206608 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:15:14.208506 12412 raft_consensus.cc:1261] T b3b86da363684ad4b531e8384b484895 P 90f427b29c2843a881082211e13781c3 [term 1 FOLLOWER]: Refusing update from remote peer c8d45d28b3e74bffbee8034f07176e4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:14.208506 12556 raft_consensus.cc:1261] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 1 FOLLOWER]: Refusing update from remote peer c8d45d28b3e74bffbee8034f07176e4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:14.208890 12782 consensus_queue.cc:1035] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d8fece9efae4563948d190e71074028" member_type: VOTER last_known_addr { host: "127.22.2.130" port: 44405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:14.209030 12779 consensus_queue.cc:1035] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f427b29c2843a881082211e13781c3" member_type: VOTER last_known_addr { host: "127.22.2.129" port: 37979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:14.211850 12787 mvcc.cc:205] Tried to move back new op lower bound from 7316165894995529728 to 7316165894940524544. Current Snapshot: MvccSnapshot[applied={T|T < 7316165894995529728}]
I20260808 07:15:14.212173 12786 mvcc.cc:205] Tried to move back new op lower bound from 7316165894995529728 to 7316165894940524544. Current Snapshot: MvccSnapshot[applied={T|T < 7316165894995529728}]
I20260808 07:15:14.213694 12256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:14.213783 12256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:15:14.214521 12256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea770ed8c3d949f9ba96ee96593ca78b: Sending DeleteTablet for 3 replicas of tablet b3b86da363684ad4b531e8384b484895
I20260808 07:15:14.214888 12533 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b86da363684ad4b531e8384b484895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:14 UTC) from {username='slave'} at 127.0.0.1:36102
I20260808 07:15:14.214943 12677 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b86da363684ad4b531e8384b484895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:14 UTC) from {username='slave'} at 127.0.0.1:40550
I20260808 07:15:14.215174 12794 tablet_replica.cc:330] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: stopping tablet replica
I20260808 07:15:14.215179 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 12320
I20260808 07:15:14.215173 12390 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b86da363684ad4b531e8384b484895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:15:14 UTC) from {username='slave'} at 127.0.0.1:37170
I20260808 07:15:14.215348 12794 raft_consensus.cc:2229] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:15:14.215500 12794 raft_consensus.cc:2258] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:14.217236 12794 ts_tablet_manager.cc:1905] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:14.219352 12794 ts_tablet_manager.cc:1918] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:14.219432 12794 log.cc:1196] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-1/wal/wals/b3b86da363684ad4b531e8384b484895
I20260808 07:15:14.219803 12794 ts_tablet_manager.cc:1939] T b3b86da363684ad4b531e8384b484895 P 7d8fece9efae4563948d190e71074028: Deleting consensus metadata
W20260808 07:15:14.222606 12239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.2.129:37979 (error 108)
I20260808 07:15:14.222824 12242 catalog_manager.cc:4841] TS 7d8fece9efae4563948d190e71074028 (127.22.2.130:44405): tablet b3b86da363684ad4b531e8384b484895 (table test-table [id=ee96dcc1601a45ab8cbf84be1f49371c]) successfully deleted
I20260808 07:15:14.223199 12769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.222997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.129:37979 (local address 127.0.0.1:37178)
0808 07:15:14.223128 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.223173 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260808 07:15:14.223317 12239 catalog_manager.cc:4584] TS 90f427b29c2843a881082211e13781c3 (127.22.2.129:37979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3b86da363684ad4b531e8384b484895: Network error: Client connection negotiation failed: client connection to 127.22.2.129:37979: connect: Connection refused (error 111)
I20260808 07:15:14.223524 12795 tablet_replica.cc:330] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: stopping tablet replica
I20260808 07:15:14.223523 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 12464
I20260808 07:15:14.223652 12795 raft_consensus.cc:2229] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:15:14.227979 12795 raft_consensus.cc:2258] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:15:14.228883 12795 ts_tablet_manager.cc:1905] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:15:14.230964 12795 ts_tablet_manager.cc:1918] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:15:14.231034 12795 log.cc:1196] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: Deleting WAL directory at /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173068742558-22538-0/minicluster-data/ts-2/wal/wals/b3b86da363684ad4b531e8384b484895
I20260808 07:15:14.231307 12795 ts_tablet_manager.cc:1939] T b3b86da363684ad4b531e8384b484895 P c8d45d28b3e74bffbee8034f07176e4e: Deleting consensus metadata
I20260808 07:15:14.232120 12239 catalog_manager.cc:4841] TS c8d45d28b3e74bffbee8034f07176e4e (127.22.2.131:34405): tablet b3b86da363684ad4b531e8384b484895 (table test-table [id=ee96dcc1601a45ab8cbf84be1f49371c]) successfully deleted
I20260808 07:15:14.234055 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 12610
I20260808 07:15:14.240682 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 12222
2026-08-08T07:15:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (675 ms)
[----------] 6 tests from EncryptionPolicyTest (9004 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:14 dist-test-slave-6k22 krb5kdc[12800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:14 dist-test-slave-6k22 krb5kdc[12800](info): set up 2 sockets
Aug 08 07:15:14 dist-test-slave-6k22 krb5kdc[12800](info): commencing operation
krb5kdc: starting...
W20260808 07:15:15.327873 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:15 dist-test-slave-6k22 krb5kdc[12800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:15.452762 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:15.537047 12814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:15.537271 12814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:15.537295 12814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:15.538807 12814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:15.538867 12814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:15.538883 12814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:15.538897 12814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:15.538908 12814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:15.540468 12814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37141
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:15.540740 12814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:15.540980 12814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:15:15.546054 12814 server_base.cc:1034] running on GCE node
W20260808 07:15:15.546123 12822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:15.546133 12820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:15.546124 12819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:15.546512 12814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:15.546782 12814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:15.547940 12814 hybrid_clock.cc:648] HybridClock initialized: now 1786173315547928 us; error 28 us; skew 500 ppm
Aug 08 07:15:15 dist-test-slave-6k22 krb5kdc[12800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173315, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:15.556052 12814 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:15.556555 12814 webserver.cc:460] Webserver started at http://127.0.0.1:33275/ using document root <none> and password file <none>
I20260808 07:15:15.556751 12814 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:15.556792 12814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:15.556895 12814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:15.557962 12814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "879e123c05b14d4b81ac2b7c28706849"
format_stamp: "Formatted at 2026-08-08 07:15:15 on dist-test-slave-6k22"
server_key: "7da8f12915f6defbbf460be66c212c1b"
server_key_iv: "1f89c0f61094b3252820506fcdb107e9"
server_key_version: "encryptionkey@0"
I20260808 07:15:15.558279 12814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "879e123c05b14d4b81ac2b7c28706849"
format_stamp: "Formatted at 2026-08-08 07:15:15 on dist-test-slave-6k22"
server_key: "7da8f12915f6defbbf460be66c212c1b"
server_key_iv: "1f89c0f61094b3252820506fcdb107e9"
server_key_version: "encryptionkey@0"
I20260808 07:15:15.559643 12814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:15:15.560381 12832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:15.560576 12814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:15.560673 12814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "879e123c05b14d4b81ac2b7c28706849"
format_stamp: "Formatted at 2026-08-08 07:15:15 on dist-test-slave-6k22"
server_key: "7da8f12915f6defbbf460be66c212c1b"
server_key_iv: "1f89c0f61094b3252820506fcdb107e9"
server_key_version: "encryptionkey@0"
I20260808 07:15:15.560750 12814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:15.581103 12814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:15.583503 12814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:15.583688 12814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:15.587728 12814 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37141
I20260808 07:15:15.587798 12891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37141 every 8 connection(s)
I20260808 07:15:15.588153 12814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:15.589004 12892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:15.591262 12892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849: Bootstrap starting.
I20260808 07:15:15.591871 12892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:15.592167 12892 log.cc:824] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:15.592869 12892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849: No bootstrap required, opened a new log
I20260808 07:15:15.594449 12892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e123c05b14d4b81ac2b7c28706849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37141 } }
I20260808 07:15:15.594586 12892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:15.594609 12892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879e123c05b14d4b81ac2b7c28706849, State: Initialized, Role: FOLLOWER
I20260808 07:15:15.594777 12892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e123c05b14d4b81ac2b7c28706849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37141 } }
I20260808 07:15:15.594841 12892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:15.594877 12892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:15.594928 12892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:15.595463 12892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e123c05b14d4b81ac2b7c28706849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37141 } }
I20260808 07:15:15.595597 12892 leader_election.cc:302] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 879e123c05b14d4b81ac2b7c28706849; no voters:
I20260808 07:15:15.595767 12892 leader_election.cc:288] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:15.595799 12895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:15.595964 12895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [term 1 LEADER]: Becoming Leader. State: Replica: 879e123c05b14d4b81ac2b7c28706849, State: Running, Role: LEADER
I20260808 07:15:15.596027 12892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:15.596056 12895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e123c05b14d4b81ac2b7c28706849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37141 } }
I20260808 07:15:15.596422 12896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "879e123c05b14d4b81ac2b7c28706849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e123c05b14d4b81ac2b7c28706849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37141 } } }
I20260808 07:15:15.596491 12896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:15.596567 12897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 879e123c05b14d4b81ac2b7c28706849. Latest consensus state: current_term: 1 leader_uuid: "879e123c05b14d4b81ac2b7c28706849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879e123c05b14d4b81ac2b7c28706849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37141 } } }
I20260808 07:15:15.596655 12897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:15.596686 12901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:15.597242 12901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:15.598245 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 12814
I20260808 07:15:15.598387 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:15.598631 22538 external_mini_cluster.cc:1428] Setting key 5782db033fdcf4d1956c21cc460b0631
I20260808 07:15:15.599000 12901 catalog_manager.cc:1347] Generated new cluster ID: 641cc8f52dd644c8b3e845b4aa0dd0d7
I20260808 07:15:15.599050 12901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:15 dist-test-slave-6k22 krb5kdc[12800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:15:15.615423 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:15.599132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55973 (local address 127.0.0.1:37141)
0808 07:15:15.603732 (+ 4600us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:15.603740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:15.603772 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:15:15.603876 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:15.603882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:15.603936 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:15.604122 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:15.604880 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.606168 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.607015 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.607462 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.607648 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:15.610241 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:15.610250 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:15.610250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:15.610261 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:15.612851 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:15.613405 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:15.613408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:15.613408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:15.613557 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:15.613936 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:15.613937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:15.613937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:15.614344 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:15.614512 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:15.614963 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 07:15:15.615099 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4544,"thread_start_us":77,"threads_started":1}
I20260808 07:15:15.616575 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:15.623879 12901 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:15.624581 12901 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:15.627524 12901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 879e123c05b14d4b81ac2b7c28706849: Generated new TSK 0
I20260808 07:15:15.627708 12901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:15.630892 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:15.618138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41065 (local address 127.0.0.1:37141)
0808 07:15:15.618179 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:15.618181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:15.623714 (+ 5533us) server_negotiation.cc:407] Connection header received
0808 07:15:15.623801 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:15.623802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:15.623828 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:15.623855 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:15.624548 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.625262 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.626313 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.626655 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.626692 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:15.628196 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:15.628198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:15.628198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:15.628205 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:15.629757 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:15.630119 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:15.630121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:15.630121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:15.630183 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:15.630384 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:15.630385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:15.630385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:15.630512 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:15.630618 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:15.630718 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:15:15.630783 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:15:15.632825 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 12814
W20260808 07:15:15.640822 12798 connection.cc:535] client connection to 127.0.0.1:37141 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37141): Connection reset by peer (error 104)
2026-08-08T07:15:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:15 dist-test-slave-6k22 krb5kdc[12926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:15 dist-test-slave-6k22 krb5kdc[12926](info): set up 2 sockets
Aug 08 07:15:15 dist-test-slave-6k22 krb5kdc[12926](info): commencing operation
krb5kdc: starting...
W20260808 07:15:16.755298 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:16 dist-test-slave-6k22 krb5kdc[12926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:16.880086 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:16.967264 12939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:16.967515 12939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:16.967558 12939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:16.968755 12939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:16.968832 12939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:16.968859 12939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:16.968890 12939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:16.968914 12939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:16.970443 12939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:16.970722 12939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:16.970986 12939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:16.976127 12944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:16.976109 12945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:16.976110 12948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:16.976143 12939 server_base.cc:1034] running on GCE node
I20260808 07:15:16.976622 12939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:16.976886 12939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:16.978029 12939 hybrid_clock.cc:648] HybridClock initialized: now 1786173316978019 us; error 31 us; skew 500 ppm
Aug 08 07:15:16 dist-test-slave-6k22 krb5kdc[12926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173316, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:16.980593 12939 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:16.981072 12939 webserver.cc:460] Webserver started at http://127.0.0.1:33247/ using document root <none> and password file <none>
I20260808 07:15:16.981310 12939 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:16.981355 12939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:16.981516 12939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:16.982604 12939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "a049c78e501f45d5a2abf228cfaabf52"
format_stamp: "Formatted at 2026-08-08 07:15:16 on dist-test-slave-6k22"
server_key: "51db726ebc9fc8d3af204dcb94b9dd4b"
server_key_iv: "d6ecb7ad8b636f14db14f601508f2a49"
server_key_version: "encryptionkey@0"
I20260808 07:15:16.982966 12939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "a049c78e501f45d5a2abf228cfaabf52"
format_stamp: "Formatted at 2026-08-08 07:15:16 on dist-test-slave-6k22"
server_key: "51db726ebc9fc8d3af204dcb94b9dd4b"
server_key_iv: "d6ecb7ad8b636f14db14f601508f2a49"
server_key_version: "encryptionkey@0"
I20260808 07:15:16.984412 12939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:16.985172 12956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:16.985342 12939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:16.985430 12939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "a049c78e501f45d5a2abf228cfaabf52"
format_stamp: "Formatted at 2026-08-08 07:15:16 on dist-test-slave-6k22"
server_key: "51db726ebc9fc8d3af204dcb94b9dd4b"
server_key_iv: "d6ecb7ad8b636f14db14f601508f2a49"
server_key_version: "encryptionkey@0"
I20260808 07:15:16.985504 12939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:16.996572 12939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:16.998422 12939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:16.998584 12939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:17.002893 12939 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42809
I20260808 07:15:17.002956 13013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42809 every 8 connection(s)
I20260808 07:15:17.003290 12939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:17.004220 13014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:17.005416 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 12939
I20260808 07:15:17.005543 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:17.005772 22538 external_mini_cluster.cc:1428] Setting key 7bf1584496b5e2f9850a67e1be93f761
I20260808 07:15:17.007124 13014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52: Bootstrap starting.
I20260808 07:15:17.007886 13014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:17.008203 13014 log.cc:824] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:17.008955 13014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52: No bootstrap required, opened a new log
I20260808 07:15:17.011356 13014 raft_consensus.cc:348] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a049c78e501f45d5a2abf228cfaabf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } }
I20260808 07:15:17.011535 13014 raft_consensus.cc:374] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:17.011638 13014 raft_consensus.cc:729] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a049c78e501f45d5a2abf228cfaabf52, State: Initialized, Role: FOLLOWER
I20260808 07:15:17.011816 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:17.006283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57071 (local address 127.0.0.1:42809)
0808 07:15:17.006458 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:17.006466 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:17.006491 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:15:17.006910 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:17.006915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:17.006959 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:17.007074 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:17.007352 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:17.008761 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:17.009572 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:17.010260 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:17.010385 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:17.011114 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:17.011119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:17.011119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:17.011321 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:17.011373 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:17.011389 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:17.011478 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:17.011585 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 07:15:17.011616 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260808 07:15:17.011795 13014 consensus_queue.cc:260] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a049c78e501f45d5a2abf228cfaabf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } }
I20260808 07:15:17.011891 13014 raft_consensus.cc:388] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:17.011927 13014 raft_consensus.cc:482] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:17.011969 13014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:17.012653 13014 raft_consensus.cc:504] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a049c78e501f45d5a2abf228cfaabf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } }
I20260808 07:15:17.012769 13014 leader_election.cc:302] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a049c78e501f45d5a2abf228cfaabf52; no voters:
I20260808 07:15:17.012917 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:17.012939 13014 leader_election.cc:288] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:17.013000 13020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:17.013170 13014 sys_catalog.cc:564] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:17.013175 13020 raft_consensus.cc:686] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [term 1 LEADER]: Becoming Leader. State: Replica: a049c78e501f45d5a2abf228cfaabf52, State: Running, Role: LEADER
I20260808 07:15:17.013296 13020 consensus_queue.cc:237] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a049c78e501f45d5a2abf228cfaabf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } }
I20260808 07:15:17.013720 13021 sys_catalog.cc:455] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a049c78e501f45d5a2abf228cfaabf52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a049c78e501f45d5a2abf228cfaabf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } } }
I20260808 07:15:17.013867 13021 sys_catalog.cc:458] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:17.013728 13022 sys_catalog.cc:455] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a049c78e501f45d5a2abf228cfaabf52. Latest consensus state: current_term: 1 leader_uuid: "a049c78e501f45d5a2abf228cfaabf52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a049c78e501f45d5a2abf228cfaabf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } } }
I20260808 07:15:17.014379 13022 sys_catalog.cc:458] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:17.014917 13032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:17.015419 13032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:17.016966 13032 catalog_manager.cc:1347] Generated new cluster ID: fd683ce1a89d47ad960cf5bec28c537d
I20260808 07:15:17.017014 13032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:17.018235 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:17.014779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56671 (local address 127.0.0.1:42809)
0808 07:15:17.014816 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:17.014818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:17.014825 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:17.014844 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:17.014845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:17.014854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:17.014904 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:17.015463 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:17.016114 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:17.017078 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:17.017284 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:17.017358 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:17.017993 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:17.017994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:17.017994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:17.017999 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:17.018011 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:17.018016 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:17.018069 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:17.018131 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 07:15:17.018140 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 07:15:17.035090 13032 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:17.035655 13032 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:17.039508 13032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a049c78e501f45d5a2abf228cfaabf52: Generated new TSK 0
I20260808 07:15:17.039738 13032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:17.042999 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 12939
W20260808 07:15:17.051646 12924 connection.cc:535] client connection to 127.0.0.1:42809 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42809): Connection reset by peer (error 104)
2026-08-08T07:15:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:17 dist-test-slave-6k22 krb5kdc[13050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:17 dist-test-slave-6k22 krb5kdc[13050](info): set up 2 sockets
Aug 08 07:15:17 dist-test-slave-6k22 krb5kdc[13050](info): commencing operation
krb5kdc: starting...
W20260808 07:15:18.177330 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:18 dist-test-slave-6k22 krb5kdc[13050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:18.300243 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:18.382855 13063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:18.383077 13063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:18.383108 13063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:18.384289 13063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:18.384341 13063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:18.384356 13063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:18.384369 13063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:18.384436 13063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:18.385922 13063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38009
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:18.386183 13063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:18.386423 13063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:18.391342 13070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:18.391410 13063 server_base.cc:1034] running on GCE node
W20260808 07:15:18.391346 13069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:18.391384 13072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:18.391868 13063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:18.392122 13063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:18.393296 13063 hybrid_clock.cc:648] HybridClock initialized: now 1786173318393283 us; error 31 us; skew 500 ppm
Aug 08 07:15:18 dist-test-slave-6k22 krb5kdc[13050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173318, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:18.395848 13063 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:18.396245 13063 webserver.cc:460] Webserver started at http://127.0.0.1:37871/ using document root <none> and password file <none>
I20260808 07:15:18.396461 13063 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:18.396502 13063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:18.396590 13063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:18.397629 13063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "bdf86fb6ad954e189b050117f74cb26b"
format_stamp: "Formatted at 2026-08-08 07:15:18 on dist-test-slave-6k22"
server_key: "202b64b142319c58fa6c47c04d5dd073"
server_key_iv: "dc1ffc43a1ccbe07bf0e8b4e8e6df84d"
server_key_version: "encryptionkey@0"
I20260808 07:15:18.397933 13063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "bdf86fb6ad954e189b050117f74cb26b"
format_stamp: "Formatted at 2026-08-08 07:15:18 on dist-test-slave-6k22"
server_key: "202b64b142319c58fa6c47c04d5dd073"
server_key_iv: "dc1ffc43a1ccbe07bf0e8b4e8e6df84d"
server_key_version: "encryptionkey@0"
I20260808 07:15:18.399178 13063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:18.399887 13079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:18.400048 13063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:18.400132 13063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "bdf86fb6ad954e189b050117f74cb26b"
format_stamp: "Formatted at 2026-08-08 07:15:18 on dist-test-slave-6k22"
server_key: "202b64b142319c58fa6c47c04d5dd073"
server_key_iv: "dc1ffc43a1ccbe07bf0e8b4e8e6df84d"
server_key_version: "encryptionkey@0"
I20260808 07:15:18.400192 13063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:18.403348 13063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:18.405303 13063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:18.405412 13063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:18.409399 13063 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38009
I20260808 07:15:18.409480 13138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38009 every 8 connection(s)
I20260808 07:15:18.409783 13063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:18.410776 13139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:18.413085 13139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b: Bootstrap starting.
I20260808 07:15:18.413661 13139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:18.413954 13139 log.cc:824] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:18.414636 13139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b: No bootstrap required, opened a new log
I20260808 07:15:18.415778 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 13063
I20260808 07:15:18.415922 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:18.416119 22538 external_mini_cluster.cc:1428] Setting key 0a014e9b681bb672d0466dea6777fa59
I20260808 07:15:18.416677 13139 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf86fb6ad954e189b050117f74cb26b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38009 } }
I20260808 07:15:18.416882 13139 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:18.416925 13139 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf86fb6ad954e189b050117f74cb26b, State: Initialized, Role: FOLLOWER
I20260808 07:15:18.417050 13139 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf86fb6ad954e189b050117f74cb26b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38009 } }
I20260808 07:15:18.417127 13139 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:18.417172 13139 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:18.417225 13139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:18.418537 13139 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf86fb6ad954e189b050117f74cb26b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38009 } }
I20260808 07:15:18.418645 13139 leader_election.cc:302] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdf86fb6ad954e189b050117f74cb26b; no voters:
I20260808 07:15:18.418777 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:18.416678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52883 (local address 127.0.0.1:38009)
0808 07:15:18.416895 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:18.416904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:18.416940 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:15:18.417052 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:18.417059 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:18.417089 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:18.417234 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:18.418270 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:18.418273 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:18.418273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:18.418277 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:18.418320 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:18.418335 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:18.418402 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:18.418494 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 07:15:18.418542 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
I20260808 07:15:18.418826 13139 leader_election.cc:288] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:18.418875 13146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:18.419001 13146 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [term 1 LEADER]: Becoming Leader. State: Replica: bdf86fb6ad954e189b050117f74cb26b, State: Running, Role: LEADER
I20260808 07:15:18.419096 13139 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:18.419086 13146 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf86fb6ad954e189b050117f74cb26b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38009 } }
I20260808 07:15:18.419413 13146 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdf86fb6ad954e189b050117f74cb26b. Latest consensus state: current_term: 1 leader_uuid: "bdf86fb6ad954e189b050117f74cb26b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf86fb6ad954e189b050117f74cb26b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38009 } } }
I20260808 07:15:18.419490 13146 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:18.419440 13147 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdf86fb6ad954e189b050117f74cb26b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf86fb6ad954e189b050117f74cb26b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38009 } } }
I20260808 07:15:18.419682 13147 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:18.419732 13153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:18.419836 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:18.420266 13153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:18.422303 13153 catalog_manager.cc:1347] Generated new cluster ID: f60295ae5dac4b3a99d7f16db6ec9fe9
I20260808 07:15:18.422361 13153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:18.422925 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:18.421808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33261 (local address 127.0.0.1:38009)
0808 07:15:18.421845 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:18.421847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:18.422007 (+ 160us) server_negotiation.cc:407] Connection header received
0808 07:15:18.422082 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:18.422083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:18.422088 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:18.422139 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:18.422716 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:18.422717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:18.422718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:18.422722 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:18.422737 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:18.422741 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:18.422787 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:18.422844 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:15:18.422859 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:15:18.431052 13153 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:18.431226 13153 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:18.436812 13153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdf86fb6ad954e189b050117f74cb26b: Generated new TSK 0
I20260808 07:15:18.437032 13153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:18.442513 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 13063
2026-08-08T07:15:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:18 dist-test-slave-6k22 krb5kdc[13175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:18 dist-test-slave-6k22 krb5kdc[13175](info): set up 2 sockets
Aug 08 07:15:18 dist-test-slave-6k22 krb5kdc[13175](info): commencing operation
krb5kdc: starting...
W20260808 07:15:19.577091 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:19 dist-test-slave-6k22 krb5kdc[13175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:19.707052 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:19.790413 13190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:19.790616 13190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:19.790637 13190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:19.791828 13190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:19.791872 13190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:19.791885 13190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:19.791898 13190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:19.791909 13190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:19.793435 13190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:19.793740 13190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:19.793990 13190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:19.799239 13195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:19.799288 13198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:19.799263 13196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:19.799571 13190 server_base.cc:1034] running on GCE node
I20260808 07:15:19.799871 13190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:19.800177 13190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:19.801322 13190 hybrid_clock.cc:648] HybridClock initialized: now 1786173319801306 us; error 26 us; skew 500 ppm
Aug 08 07:15:19 dist-test-slave-6k22 krb5kdc[13175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173319, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:19.803877 13190 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:19.804360 13190 webserver.cc:460] Webserver started at http://127.0.0.1:32997/ using document root <none> and password file <none>
I20260808 07:15:19.804606 13190 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:19.804674 13190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:19.804829 13190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:19.806036 13190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "793230d68b4b434b83078be603716765"
format_stamp: "Formatted at 2026-08-08 07:15:19 on dist-test-slave-6k22"
server_key: "0f10dc2c680988fb6d4856ed35ecc353"
server_key_iv: "4fbaed352cc348f312c5bea744ebff0e"
server_key_version: "encryptionkey@0"
I20260808 07:15:19.806521 13190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "793230d68b4b434b83078be603716765"
format_stamp: "Formatted at 2026-08-08 07:15:19 on dist-test-slave-6k22"
server_key: "0f10dc2c680988fb6d4856ed35ecc353"
server_key_iv: "4fbaed352cc348f312c5bea744ebff0e"
server_key_version: "encryptionkey@0"
I20260808 07:15:19.807915 13190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:19.808660 13205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:19.808846 13190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:19.808945 13190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "793230d68b4b434b83078be603716765"
format_stamp: "Formatted at 2026-08-08 07:15:19 on dist-test-slave-6k22"
server_key: "0f10dc2c680988fb6d4856ed35ecc353"
server_key_iv: "4fbaed352cc348f312c5bea744ebff0e"
server_key_version: "encryptionkey@0"
I20260808 07:15:19.809033 13190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:19.819698 13190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:19.821465 13190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:19.821624 13190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:19.825865 13190 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39017
I20260808 07:15:19.825897 13264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39017 every 8 connection(s)
I20260808 07:15:19.826251 13190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:19.827126 13265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:19.829433 13265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765: Bootstrap starting.
I20260808 07:15:19.829973 13265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:19.830211 13265 log.cc:824] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:19.830871 13265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765: No bootstrap required, opened a new log
I20260808 07:15:19.832597 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 13190
I20260808 07:15:19.832535 13265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793230d68b4b434b83078be603716765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } }
I20260808 07:15:19.832729 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:19.832722 13265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:19.832762 13265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 793230d68b4b434b83078be603716765, State: Initialized, Role: FOLLOWER
I20260808 07:15:19.832929 22538 external_mini_cluster.cc:1428] Setting key 253af6064223a2d147627cc71fc6e979
I20260808 07:15:19.832931 13265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793230d68b4b434b83078be603716765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } }
I20260808 07:15:19.833009 13265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:19.833053 13265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:19.833110 13265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:19.833824 13265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793230d68b4b434b83078be603716765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } }
I20260808 07:15:19.833951 13265 leader_election.cc:302] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 793230d68b4b434b83078be603716765; no voters:
I20260808 07:15:19.834110 13265 leader_election.cc:288] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:19.834139 13270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:19.834283 13270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [term 1 LEADER]: Becoming Leader. State: Replica: 793230d68b4b434b83078be603716765, State: Running, Role: LEADER
I20260808 07:15:19.834378 13265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:19.834383 13270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793230d68b4b434b83078be603716765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } }
I20260808 07:15:19.834759 13270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 793230d68b4b434b83078be603716765. Latest consensus state: current_term: 1 leader_uuid: "793230d68b4b434b83078be603716765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793230d68b4b434b83078be603716765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } } }
I20260808 07:15:19.834867 13270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:19.834774 13271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "793230d68b4b434b83078be603716765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793230d68b4b434b83078be603716765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } } }
I20260808 07:15:19.835031 13271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:19.835081 13279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:19.835628 13279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:19.837198 13279 catalog_manager.cc:1347] Generated new cluster ID: dcb4d68344b142c288e1fa4e55442c93
I20260808 07:15:19.837244 13279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:19 dist-test-slave-6k22 krb5kdc[13175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:15:19.845498 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:19.833436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51253 (local address 127.0.0.1:39017)
0808 07:15:19.833964 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:19.833974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:19.834009 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:15:19.834140 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:19.834148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:19.834207 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:19.835236 (+ 1029us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:19.835374 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:19.836826 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:19.837907 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:19.838542 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:19.838653 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:19.841554 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:19.841559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:19.841559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:19.841572 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:19.843801 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:19.844193 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:19.844197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:19.844197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:19.844325 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:19.844536 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:19.844537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:19.844537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:19.844868 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:19.845027 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:19.845100 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 07:15:19.845249 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":447,"threads_started":1}
I20260808 07:15:19.846736 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:19.856381 13279 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:19.856918 13279 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:19.859115 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:19.848298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57963 (local address 127.0.0.1:39017)
0808 07:15:19.848345 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:19.848347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:19.851759 (+ 3412us) server_negotiation.cc:407] Connection header received
0808 07:15:19.851814 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:19.851815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:19.851827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:19.851881 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:19.852500 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:19.853264 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:19.854226 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:19.854568 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:19.854640 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:19.856045 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:19.856047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:19.856047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:19.856054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:19.857798 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:19.858168 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:19.858169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:19.858169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:19.858229 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:19.858445 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:19.858446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:19.858446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:19.858577 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:19.858675 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:19.858791 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 07:15:19.859023 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:19.861261 13279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 793230d68b4b434b83078be603716765: Generated new TSK 0
I20260808 07:15:19.861433 13279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:19.866988 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 13190
2026-08-08T07:15:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:19 dist-test-slave-6k22 krb5kdc[13298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:19 dist-test-slave-6k22 krb5kdc[13298](info): set up 2 sockets
Aug 08 07:15:19 dist-test-slave-6k22 krb5kdc[13298](info): commencing operation
krb5kdc: starting...
W20260808 07:15:20.968984 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:21 dist-test-slave-6k22 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:21.094610 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:21.176826 13311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:21.177070 13311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:21.177107 13311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:21.178220 13311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:21.178287 13311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:21.178310 13311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:21.178331 13311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:21.178401 13311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:21.179903 13311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:21.180173 13311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:21.180401 13311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:21.185189 13320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:21.185262 13318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:21.185379 13311 server_base.cc:1034] running on GCE node
W20260808 07:15:21.185199 13317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:21.185719 13311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:21.186043 13311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:21.187182 13311 hybrid_clock.cc:648] HybridClock initialized: now 1786173321187165 us; error 37 us; skew 500 ppm
Aug 08 07:15:21 dist-test-slave-6k22 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:21.194542 13311 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:21.195024 13311 webserver.cc:460] Webserver started at http://127.0.0.1:46811/ using document root <none> and password file <none>
I20260808 07:15:21.195271 13311 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:21.195319 13311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:21.195492 13311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:21.196586 13311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "9d8c3c1ca28d437da9927e09fb0cf535"
format_stamp: "Formatted at 2026-08-08 07:15:21 on dist-test-slave-6k22"
server_key: "f413652746701fbf966bb80112524dca"
server_key_iv: "e24d72e99c3fafec2cfbe3321160fd6c"
server_key_version: "encryptionkey@0"
I20260808 07:15:21.196946 13311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "9d8c3c1ca28d437da9927e09fb0cf535"
format_stamp: "Formatted at 2026-08-08 07:15:21 on dist-test-slave-6k22"
server_key: "f413652746701fbf966bb80112524dca"
server_key_iv: "e24d72e99c3fafec2cfbe3321160fd6c"
server_key_version: "encryptionkey@0"
I20260808 07:15:21.198274 13311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:21.199002 13329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:21.199200 13311 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:21.199285 13311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "9d8c3c1ca28d437da9927e09fb0cf535"
format_stamp: "Formatted at 2026-08-08 07:15:21 on dist-test-slave-6k22"
server_key: "f413652746701fbf966bb80112524dca"
server_key_iv: "e24d72e99c3fafec2cfbe3321160fd6c"
server_key_version: "encryptionkey@0"
I20260808 07:15:21.199340 13311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:21.217270 13311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:21.219095 13311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:21.219206 13311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:21.223310 13311 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34467
I20260808 07:15:21.223393 13385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34467 every 8 connection(s)
I20260808 07:15:21.223755 13311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:21.224649 13386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:21.226903 13386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535: Bootstrap starting.
I20260808 07:15:21.227478 13386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:21.227850 13386 log.cc:824] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:21.228636 13386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535: No bootstrap required, opened a new log
I20260808 07:15:21.230180 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 13311
I20260808 07:15:21.230320 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:21.230518 22538 external_mini_cluster.cc:1428] Setting key de394f0d6c5a3595bc41922b387867e0
I20260808 07:15:21.230477 13386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8c3c1ca28d437da9927e09fb0cf535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } }
I20260808 07:15:21.230654 13386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:21.230728 13386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d8c3c1ca28d437da9927e09fb0cf535, State: Initialized, Role: FOLLOWER
I20260808 07:15:21.230902 13386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8c3c1ca28d437da9927e09fb0cf535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } }
I20260808 07:15:21.230993 13386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:21.231034 13386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:21.231086 13386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:21.231848 13386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8c3c1ca28d437da9927e09fb0cf535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } }
I20260808 07:15:21.231956 13386 leader_election.cc:302] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d8c3c1ca28d437da9927e09fb0cf535; no voters:
I20260808 07:15:21.232116 13386 leader_election.cc:288] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:21.232141 13391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:21.232251 13391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [term 1 LEADER]: Becoming Leader. State: Replica: 9d8c3c1ca28d437da9927e09fb0cf535, State: Running, Role: LEADER
I20260808 07:15:21.232331 13391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8c3c1ca28d437da9927e09fb0cf535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } }
I20260808 07:15:21.232379 13386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:21.232659 13392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d8c3c1ca28d437da9927e09fb0cf535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8c3c1ca28d437da9927e09fb0cf535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } } }
I20260808 07:15:21.232750 13392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:21.232946 13393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d8c3c1ca28d437da9927e09fb0cf535. Latest consensus state: current_term: 1 leader_uuid: "9d8c3c1ca28d437da9927e09fb0cf535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8c3c1ca28d437da9927e09fb0cf535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } } }
I20260808 07:15:21.233006 13393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:21.233225 13396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:21.233685 13396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:21.235636 13396 catalog_manager.cc:1347] Generated new cluster ID: 56180fd6c34844f79d3a71dc7a71e938
I20260808 07:15:21.235690 13396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:21.236922 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:21.231031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44961 (local address 127.0.0.1:34467)
0808 07:15:21.231220 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:21.231229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:21.231297 (+ 68us) server_negotiation.cc:407] Connection header received
0808 07:15:21.231769 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:21.231779 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:21.231845 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:21.231994 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:21.232219 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:21.233663 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:21.234784 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:21.235380 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:21.235493 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:21.236246 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:21.236250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:21.236251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:21.236454 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:21.236483 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:21.236494 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:21.236574 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:21.236651 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:15:21.236715 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260808 07:15:21.237775 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:21.242323 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:21.239075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56651 (local address 127.0.0.1:34467)
0808 07:15:21.239112 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:21.239113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:21.239272 (+ 159us) server_negotiation.cc:407] Connection header received
0808 07:15:21.239328 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:21.239328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:21.239346 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:21.239389 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:21.239534 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:21.240432 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:21.241140 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:21.241367 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:21.241452 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:21.242091 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:21.242092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:21.242092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:21.242098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:21.242112 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:21.242117 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:21.242159 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:21.242224 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:15:21.242247 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:15:21.250379 13396 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:21.250903 13396 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:21.259181 13396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d8c3c1ca28d437da9927e09fb0cf535: Generated new TSK 0
I20260808 07:15:21.259378 13396 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:21.265440 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 13311
2026-08-08T07:15:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:21 dist-test-slave-6k22 krb5kdc[13421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:21 dist-test-slave-6k22 krb5kdc[13421](info): set up 2 sockets
Aug 08 07:15:21 dist-test-slave-6k22 krb5kdc[13421](info): commencing operation
krb5kdc: starting...
W20260808 07:15:22.410774 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:22 dist-test-slave-6k22 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:22.538769 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:22.621397 13434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:22.621613 13434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:22.621636 13434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:22.622718 13434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:22.622752 13434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:22.622767 13434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:22.622781 13434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:22.622792 13434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:22.624234 13434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:22.624465 13434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:22.624637 13434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:22.629621 13442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:22.629662 13444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:22.629684 13434 server_base.cc:1034] running on GCE node
W20260808 07:15:22.629635 13440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:22.630133 13434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:22.630470 13434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:22.631620 13434 hybrid_clock.cc:648] HybridClock initialized: now 1786173322631610 us; error 33 us; skew 500 ppm
Aug 08 07:15:22 dist-test-slave-6k22 krb5kdc[13421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:22.634064 13434 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:22.634508 13434 webserver.cc:460] Webserver started at http://127.0.0.1:32851/ using document root <none> and password file <none>
I20260808 07:15:22.634735 13434 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:22.634801 13434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:22.634943 13434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:22.636081 13434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "4521cf5fb9f049dcb20329acfecf4084"
format_stamp: "Formatted at 2026-08-08 07:15:22 on dist-test-slave-6k22"
server_key: "8819b20b86887410cbaeda18ee3273f8"
server_key_iv: "31abc3724e4a23215df3b3aaee631860"
server_key_version: "encryptionkey@0"
I20260808 07:15:22.636458 13434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "4521cf5fb9f049dcb20329acfecf4084"
format_stamp: "Formatted at 2026-08-08 07:15:22 on dist-test-slave-6k22"
server_key: "8819b20b86887410cbaeda18ee3273f8"
server_key_iv: "31abc3724e4a23215df3b3aaee631860"
server_key_version: "encryptionkey@0"
I20260808 07:15:22.637794 13434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:22.638512 13451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:22.638684 13434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:22.638792 13434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "4521cf5fb9f049dcb20329acfecf4084"
format_stamp: "Formatted at 2026-08-08 07:15:22 on dist-test-slave-6k22"
server_key: "8819b20b86887410cbaeda18ee3273f8"
server_key_iv: "31abc3724e4a23215df3b3aaee631860"
server_key_version: "encryptionkey@0"
I20260808 07:15:22.638867 13434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:22.642295 13434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:22.644201 13434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:22.644362 13434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:22.648264 13434 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37143
I20260808 07:15:22.648295 13506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37143 every 8 connection(s)
I20260808 07:15:22.648648 13434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:22.649652 13507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:22.651912 13507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084: Bootstrap starting.
I20260808 07:15:22.652462 13507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:22.652710 13507 log.cc:824] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:22.653360 13507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084: No bootstrap required, opened a new log
I20260808 07:15:22.653906 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 13434
I20260808 07:15:22.654045 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:22.654227 22538 external_mini_cluster.cc:1428] Setting key a2339821aca25e3ae184f032c41859d2
I20260808 07:15:22.655505 13507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4521cf5fb9f049dcb20329acfecf4084" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } }
I20260808 07:15:22.655719 13507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:22.655777 13507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4521cf5fb9f049dcb20329acfecf4084, State: Initialized, Role: FOLLOWER
I20260808 07:15:22.655942 13507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4521cf5fb9f049dcb20329acfecf4084" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } }
I20260808 07:15:22.656050 13507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:22.656105 13507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:22.656165 13507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:22.656865 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:22.654790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45675 (local address 127.0.0.1:37143)
0808 07:15:22.655001 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:22.655012 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:22.655048 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:15:22.655463 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:22.655472 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:22.655508 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:22.655671 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:22.656379 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:22.656382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:22.656382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:22.656386 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:22.656429 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:22.656439 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:22.656495 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:22.656591 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:15:22.656640 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260808 07:15:22.657270 13507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4521cf5fb9f049dcb20329acfecf4084" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } }
I20260808 07:15:22.657399 13507 leader_election.cc:302] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4521cf5fb9f049dcb20329acfecf4084; no voters:
I20260808 07:15:22.657573 13507 leader_election.cc:288] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:22.657657 13512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:22.657778 13512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [term 1 LEADER]: Becoming Leader. State: Replica: 4521cf5fb9f049dcb20329acfecf4084, State: Running, Role: LEADER
I20260808 07:15:22.657838 13507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:22.657888 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:22.657876 13512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4521cf5fb9f049dcb20329acfecf4084" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } }
I20260808 07:15:22.658216 13513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4521cf5fb9f049dcb20329acfecf4084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4521cf5fb9f049dcb20329acfecf4084" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } } }
I20260808 07:15:22.658324 13513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:22.658202 13512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4521cf5fb9f049dcb20329acfecf4084. Latest consensus state: current_term: 1 leader_uuid: "4521cf5fb9f049dcb20329acfecf4084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4521cf5fb9f049dcb20329acfecf4084" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37143 } } }
I20260808 07:15:22.658527 13512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:22.658654 13519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:22.659157 13519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:22.660778 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:22.659623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45137 (local address 127.0.0.1:37143)
0808 07:15:22.659659 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:22.659661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:22.659852 (+ 191us) server_negotiation.cc:407] Connection header received
0808 07:15:22.659924 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:22.659925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:22.659931 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:22.659985 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:22.660543 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:22.660544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:22.660544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:22.660548 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:22.660561 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:22.660565 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:22.660597 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:22.660703 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:15:22.660716 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:22.660876 13519 catalog_manager.cc:1347] Generated new cluster ID: 0d5d0c2d0969442cbfb700939d9ebbed
I20260808 07:15:22.660919 13519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:22.668687 13519 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:22.668880 13519 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:22.672870 13519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4521cf5fb9f049dcb20329acfecf4084: Generated new TSK 0
I20260808 07:15:22.673050 13519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:22.681056 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 13434
2026-08-08T07:15:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:22 dist-test-slave-6k22 krb5kdc[13538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:22 dist-test-slave-6k22 krb5kdc[13538](info): set up 2 sockets
Aug 08 07:15:22 dist-test-slave-6k22 krb5kdc[13538](info): commencing operation
krb5kdc: starting...
W20260808 07:15:23.807060 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:23 dist-test-slave-6k22 krb5kdc[13538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:23.934522 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35495
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:24.020644 13551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:24.020893 13551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:24.020920 13551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:24.022378 13551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:24.022437 13551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:24.022452 13551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:24.022465 13551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:24.022478 13551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:24.023957 13551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:24.024166 13551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:24.024361 13551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:24.029264 13561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:24.029274 13558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:24.029503 13551 server_base.cc:1034] running on GCE node
W20260808 07:15:24.029292 13559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:24.029846 13551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:24.030113 13551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:24.031258 13551 hybrid_clock.cc:648] HybridClock initialized: now 1786173324031238 us; error 38 us; skew 500 ppm
Aug 08 07:15:24 dist-test-slave-6k22 krb5kdc[13538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173324, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:24.033927 13551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:24.034317 13551 webserver.cc:460] Webserver started at http://127.0.0.1:44489/ using document root <none> and password file <none>
I20260808 07:15:24.034592 13551 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:24.034641 13551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:24.034816 13551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:24.035952 13551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "755d0917947840d4b04329a052f0fde1"
format_stamp: "Formatted at 2026-08-08 07:15:24 on dist-test-slave-6k22"
server_key: "fa9a3b5857cc7409787c090f099a3300"
server_key_iv: "e72a7f233cd912af4112c647f0deefb4"
server_key_version: "encryptionkey@0"
I20260808 07:15:24.036301 13551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "755d0917947840d4b04329a052f0fde1"
format_stamp: "Formatted at 2026-08-08 07:15:24 on dist-test-slave-6k22"
server_key: "fa9a3b5857cc7409787c090f099a3300"
server_key_iv: "e72a7f233cd912af4112c647f0deefb4"
server_key_version: "encryptionkey@0"
I20260808 07:15:24.037634 13551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:24.038342 13570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:24.038550 13551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:24.038646 13551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "755d0917947840d4b04329a052f0fde1"
format_stamp: "Formatted at 2026-08-08 07:15:24 on dist-test-slave-6k22"
server_key: "fa9a3b5857cc7409787c090f099a3300"
server_key_iv: "e72a7f233cd912af4112c647f0deefb4"
server_key_version: "encryptionkey@0"
I20260808 07:15:24.038719 13551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:24.060547 13551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:24.062541 13551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:24.062701 13551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:24.067063 13551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35495
I20260808 07:15:24.067121 13624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35495 every 8 connection(s)
I20260808 07:15:24.067479 13551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:24.068430 13625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:24.070189 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 13551
I20260808 07:15:24.070320 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:24.070523 22538 external_mini_cluster.cc:1428] Setting key d0b011727de65e235256232523b0192a
I20260808 07:15:24.071172 13625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1: Bootstrap starting.
I20260808 07:15:24.071877 13625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:24.072180 13625 log.cc:824] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:24.072882 13625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1: No bootstrap required, opened a new log
I20260808 07:15:24.074661 13625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755d0917947840d4b04329a052f0fde1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35495 } }
I20260808 07:15:24.074827 13625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:24.074880 13625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755d0917947840d4b04329a052f0fde1, State: Initialized, Role: FOLLOWER
I20260808 07:15:24.075019 13625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755d0917947840d4b04329a052f0fde1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35495 } }
I20260808 07:15:24.075115 13625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:24.075147 13625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:24.075245 13625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:24.076028 13625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755d0917947840d4b04329a052f0fde1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35495 } }
I20260808 07:15:24.076150 13625 leader_election.cc:302] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 755d0917947840d4b04329a052f0fde1; no voters:
I20260808 07:15:24.076342 13625 leader_election.cc:288] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:24.076407 13630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:24.076568 13630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [term 1 LEADER]: Becoming Leader. State: Replica: 755d0917947840d4b04329a052f0fde1, State: Running, Role: LEADER
I20260808 07:15:24.076624 13625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:24.076699 13630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755d0917947840d4b04329a052f0fde1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35495 } }
I20260808 07:15:24.077178 13630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 755d0917947840d4b04329a052f0fde1. Latest consensus state: current_term: 1 leader_uuid: "755d0917947840d4b04329a052f0fde1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755d0917947840d4b04329a052f0fde1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35495 } } }
I20260808 07:15:24.077201 13631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "755d0917947840d4b04329a052f0fde1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755d0917947840d4b04329a052f0fde1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35495 } } }
I20260808 07:15:24.077322 13630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:24.077332 13631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1 [sys.catalog]: This master's current role is: LEADER
Aug 08 07:15:24 dist-test-slave-6k22 krb5kdc[13538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:15:24.077685 13641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:24.078538 13641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:24.080693 13641 catalog_manager.cc:1347] Generated new cluster ID: 147824a4642748e183635f106532adf1
I20260808 07:15:24.080772 13641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:24.082921 13628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:24.071088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:44091 (local address 127.0.0.1:35495)
0808 07:15:24.071273 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:24.071284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:24.071320 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:15:24.071828 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:24.071842 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:24.071893 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:24.072084 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:24.072787 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:24.073725 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:24.074504 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:24.074992 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:24.075142 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:24.078387 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:24.078391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:24.078392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:24.078406 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:24.080707 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:24.081202 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:24.081206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:24.081206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:24.081311 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:24.081584 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:24.081585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:24.081586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:24.081892 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:24.082050 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:24.082593 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 07:15:24.082702 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260808 07:15:24.083861 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:24.093482 13628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:24.085409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:58999 (local address 127.0.0.1:35495)
0808 07:15:24.085447 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:24.085449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:24.085569 (+ 120us) server_negotiation.cc:407] Connection header received
0808 07:15:24.085644 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:24.085645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:24.085655 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:24.085720 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:24.086444 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:24.087213 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:24.088355 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:24.088678 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:24.088756 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:24.090378 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:24.090380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:24.090380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:24.090387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:24.092107 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:24.092577 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:24.092579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:24.092579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:24.092642 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:24.092897 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:24.092898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:24.092898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:24.093033 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:24.093139 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:24.093307 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:15:24.093383 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:24.097604 13641 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:24.098308 13641 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:24.114457 13641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 755d0917947840d4b04329a052f0fde1: Generated new TSK 0
I20260808 07:15:24.114692 13641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:24.120518 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 13551
W20260808 07:15:24.127189 13536 connection.cc:535] client connection to 127.0.0.1:35495 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35495): Connection reset by peer (error 104)
2026-08-08T07:15:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:24 dist-test-slave-6k22 krb5kdc[13660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:24 dist-test-slave-6k22 krb5kdc[13660](info): set up 2 sockets
Aug 08 07:15:24 dist-test-slave-6k22 krb5kdc[13660](info): commencing operation
krb5kdc: starting...
W20260808 07:15:25.228454 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:25 dist-test-slave-6k22 krb5kdc[13660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:25.352705 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:25.434561 13675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:25.434767 13675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:25.434788 13675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:25.435876 13675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:25.435909 13675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:25.435923 13675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:25.435935 13675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:25.435947 13675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:25.437320 13675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:25.437588 13675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:25.437844 13675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:25.442801 13682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:25.442894 13681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:25.443042 13675 server_base.cc:1034] running on GCE node
W20260808 07:15:25.443079 13684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:25.443331 13675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:25.443678 13675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:25.444847 13675 hybrid_clock.cc:648] HybridClock initialized: now 1786173325444833 us; error 38 us; skew 500 ppm
Aug 08 07:15:25 dist-test-slave-6k22 krb5kdc[13660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173325, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:25.447366 13675 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:25.447837 13675 webserver.cc:460] Webserver started at http://127.0.0.1:40787/ using document root <none> and password file <none>
I20260808 07:15:25.448026 13675 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:25.448065 13675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:25.448164 13675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:25.449173 13675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "ed06e9e2fdb64478ab387ea3a9e8b041"
format_stamp: "Formatted at 2026-08-08 07:15:25 on dist-test-slave-6k22"
server_key: "2b091fc295c095957f57d064f2062ecd"
server_key_iv: "109faf31485737ead0c63b6c8dbb1843"
server_key_version: "encryptionkey@0"
I20260808 07:15:25.449486 13675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "ed06e9e2fdb64478ab387ea3a9e8b041"
format_stamp: "Formatted at 2026-08-08 07:15:25 on dist-test-slave-6k22"
server_key: "2b091fc295c095957f57d064f2062ecd"
server_key_iv: "109faf31485737ead0c63b6c8dbb1843"
server_key_version: "encryptionkey@0"
I20260808 07:15:25.450759 13675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:25.451469 13692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:25.451647 13675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:25.451741 13675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "ed06e9e2fdb64478ab387ea3a9e8b041"
format_stamp: "Formatted at 2026-08-08 07:15:25 on dist-test-slave-6k22"
server_key: "2b091fc295c095957f57d064f2062ecd"
server_key_iv: "109faf31485737ead0c63b6c8dbb1843"
server_key_version: "encryptionkey@0"
I20260808 07:15:25.451799 13675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:25.474656 13675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:25.476752 13675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:25.476897 13675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:25.481149 13675 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44575
I20260808 07:15:25.481217 13744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44575 every 8 connection(s)
I20260808 07:15:25.481583 13675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:25.482492 13745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:25.484941 13745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041: Bootstrap starting.
I20260808 07:15:25.485546 13745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:25.485864 13745 log.cc:824] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:25.486658 13745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041: No bootstrap required, opened a new log
I20260808 07:15:25.488247 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 13675
I20260808 07:15:25.488360 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:25.488332 13745 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e9e2fdb64478ab387ea3a9e8b041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260808 07:15:25.488544 13745 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:25.488602 22538 external_mini_cluster.cc:1428] Setting key 012335e8bfeabfbf557dfa4ed82c04e7
I20260808 07:15:25.488595 13745 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed06e9e2fdb64478ab387ea3a9e8b041, State: Initialized, Role: FOLLOWER
I20260808 07:15:25.488772 13745 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e9e2fdb64478ab387ea3a9e8b041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260808 07:15:25.488883 13745 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:25.488930 13745 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:25.488981 13745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:25.489907 13745 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e9e2fdb64478ab387ea3a9e8b041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260808 07:15:25.490029 13745 leader_election.cc:302] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed06e9e2fdb64478ab387ea3a9e8b041; no voters:
I20260808 07:15:25.490211 13745 leader_election.cc:288] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:25.490254 13752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:25.490427 13752 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [term 1 LEADER]: Becoming Leader. State: Replica: ed06e9e2fdb64478ab387ea3a9e8b041, State: Running, Role: LEADER
I20260808 07:15:25.490487 13745 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:25.490588 13752 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e9e2fdb64478ab387ea3a9e8b041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260808 07:15:25.490967 13752 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed06e9e2fdb64478ab387ea3a9e8b041. Latest consensus state: current_term: 1 leader_uuid: "ed06e9e2fdb64478ab387ea3a9e8b041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e9e2fdb64478ab387ea3a9e8b041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } } }
I20260808 07:15:25.491043 13752 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:25.491128 13753 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed06e9e2fdb64478ab387ea3a9e8b041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed06e9e2fdb64478ab387ea3a9e8b041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } } }
I20260808 07:15:25.491242 13753 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041 [sys.catalog]: This master's current role is: LEADER
W20260808 07:15:25.492558 13767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:25.492617 13767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:25.492664 13759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:25.492801 13759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:25.494442 13759 catalog_manager.cc:1347] Generated new cluster ID: 9648393f1c8f4fb0ae8322c5e5425eb1
I20260808 07:15:25.494500 13759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:25.494673 13750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:25.489158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:37423 (local address 127.0.0.1:44575)
0808 07:15:25.489366 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:25.489376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:25.489414 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:15:25.489533 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:25.489538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:25.489590 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:25.489724 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:25.489924 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:25.491242 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:25.492455 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:25.492950 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:25.493073 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:25.493848 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:25.493851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:25.493852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:25.494095 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:25.494132 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:25.494148 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:25.494253 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:25.494414 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:15:25.494444 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":115,"threads_started":1}
I20260808 07:15:25.495637 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:25.499855 13750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:25.496956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:52341 (local address 127.0.0.1:44575)
0808 07:15:25.496994 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:25.496996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:25.497125 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:15:25.497193 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:25.497194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:25.497206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:25.497261 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:25.497408 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:25.497908 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:25.498609 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:25.498847 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:25.498922 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:25.499604 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:25.499606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:25.499606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:25.499610 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:25.499624 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:25.499628 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:25.499682 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:25.499742 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:15:25.499755 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:25.509011 13759 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:25.509585 13759 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:25.516297 13759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed06e9e2fdb64478ab387ea3a9e8b041: Generated new TSK 0
I20260808 07:15:25.516486 13759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:25.527909 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 13675
W20260808 07:15:25.535280 13658 connection.cc:535] client connection to 127.0.0.1:44575 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44575): Connection reset by peer (error 104)
2026-08-08T07:15:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:25 dist-test-slave-6k22 krb5kdc[13780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:25 dist-test-slave-6k22 krb5kdc[13780](info): set up 2 sockets
Aug 08 07:15:25 dist-test-slave-6k22 krb5kdc[13780](info): commencing operation
krb5kdc: starting...
W20260808 07:15:26.642490 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:26 dist-test-slave-6k22 krb5kdc[13780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:26.774089 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41785
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:26.858491 13796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:26.858711 13796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:26.858736 13796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:26.860029 13796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:26.860078 13796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:26.860092 13796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:26.860105 13796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:26.860117 13796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:26.861649 13796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:26.861910 13796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:26.862174 13796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:26.867311 13805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:26.867347 13803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:26.867352 13802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:26.867623 13796 server_base.cc:1034] running on GCE node
I20260808 07:15:26.867866 13796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:26.868134 13796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:26.869278 13796 hybrid_clock.cc:648] HybridClock initialized: now 1786173326869268 us; error 32 us; skew 500 ppm
Aug 08 07:15:26 dist-test-slave-6k22 krb5kdc[13780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173326, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:26.876771 13796 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:26.877187 13796 webserver.cc:460] Webserver started at http://127.0.0.1:39425/ using document root <none> and password file <none>
I20260808 07:15:26.877408 13796 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:26.877450 13796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:26.877544 13796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:26.878616 13796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "3f2ea06aa1ff414d9fd056ced81f1057"
format_stamp: "Formatted at 2026-08-08 07:15:26 on dist-test-slave-6k22"
server_key: "ce236640551ea9382addf39424b367bd"
server_key_iv: "80bf49e1bf8175d642e08de96407e747"
server_key_version: "encryptionkey@0"
I20260808 07:15:26.878928 13796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "3f2ea06aa1ff414d9fd056ced81f1057"
format_stamp: "Formatted at 2026-08-08 07:15:26 on dist-test-slave-6k22"
server_key: "ce236640551ea9382addf39424b367bd"
server_key_iv: "80bf49e1bf8175d642e08de96407e747"
server_key_version: "encryptionkey@0"
I20260808 07:15:26.880340 13796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:26.881094 13813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:26.881309 13796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:15:26.881407 13796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "3f2ea06aa1ff414d9fd056ced81f1057"
format_stamp: "Formatted at 2026-08-08 07:15:26 on dist-test-slave-6k22"
server_key: "ce236640551ea9382addf39424b367bd"
server_key_iv: "80bf49e1bf8175d642e08de96407e747"
server_key_version: "encryptionkey@0"
I20260808 07:15:26.881466 13796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:26.884776 13796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:26.886631 13796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:26.886736 13796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:26.890638 13796 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41785
I20260808 07:15:26.890698 13870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41785 every 8 connection(s)
I20260808 07:15:26.891012 13796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:26.892093 13871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:26.894327 13871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057: Bootstrap starting.
I20260808 07:15:26.894929 13871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:26.895219 13871 log.cc:824] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:26.895998 13871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057: No bootstrap required, opened a new log
I20260808 07:15:26.897595 13871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2ea06aa1ff414d9fd056ced81f1057" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41785 } }
I20260808 07:15:26.897758 13871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:26.897814 13871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2ea06aa1ff414d9fd056ced81f1057, State: Initialized, Role: FOLLOWER
I20260808 07:15:26.897965 13871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2ea06aa1ff414d9fd056ced81f1057" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41785 } }
I20260808 07:15:26.898048 13871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:26.898092 13871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:26.898142 13871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:26.898684 13871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2ea06aa1ff414d9fd056ced81f1057" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41785 } }
I20260808 07:15:26.898861 13871 leader_election.cc:302] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f2ea06aa1ff414d9fd056ced81f1057; no voters:
I20260808 07:15:26.899119 13871 leader_election.cc:288] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:26.899159 13874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:26.899340 13874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [term 1 LEADER]: Becoming Leader. State: Replica: 3f2ea06aa1ff414d9fd056ced81f1057, State: Running, Role: LEADER
I20260808 07:15:26.899418 13871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:26.899502 13874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2ea06aa1ff414d9fd056ced81f1057" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41785 } }
I20260808 07:15:26.899786 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 13796
I20260808 07:15:26.899919 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:26.900050 13876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f2ea06aa1ff414d9fd056ced81f1057. Latest consensus state: current_term: 1 leader_uuid: "3f2ea06aa1ff414d9fd056ced81f1057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2ea06aa1ff414d9fd056ced81f1057" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41785 } } }
I20260808 07:15:26.900139 13876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:26.900182 22538 external_mini_cluster.cc:1428] Setting key e4094c6a7f34831200f7d9be0e994d97
I20260808 07:15:26.900537 13875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f2ea06aa1ff414d9fd056ced81f1057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2ea06aa1ff414d9fd056ced81f1057" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41785 } } }
I20260808 07:15:26.900569 13881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:26.900686 13875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:26.901257 13881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:26.902861 13881 catalog_manager.cc:1347] Generated new cluster ID: 9dd9e19330bb4cf892add73e22023292
I20260808 07:15:26.902959 13881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:26.904529 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:26.900928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:38891 (local address 127.0.0.1:41785)
0808 07:15:26.901393 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:26.901404 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:26.902868 (+ 1464us) server_negotiation.cc:407] Connection header received
0808 07:15:26.902995 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:26.903003 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:26.903037 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:26.903191 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:26.904080 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:26.904084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:26.904084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:26.904089 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:26.904129 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:26.904144 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:26.904199 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:26.904312 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 07:15:26.904356 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":87,"threads_started":1}
I20260808 07:15:26.906672 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:26.913130 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:26.911905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:51275 (local address 127.0.0.1:41785)
0808 07:15:26.911952 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:26.911954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:26.912121 (+ 167us) server_negotiation.cc:407] Connection header received
0808 07:15:26.912190 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:26.912191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:26.912197 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:26.912245 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:26.912890 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:26.912892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:26.912892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:26.912896 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:26.912912 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:26.912916 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:26.912974 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:26.913046 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 07:15:26.913058 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:15:26.914734 13881 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:26.914912 13881 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:26.921242 13881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f2ea06aa1ff414d9fd056ced81f1057: Generated new TSK 0
I20260808 07:15:26.921461 13881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:26.924075 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 13796
2026-08-08T07:15:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:26 dist-test-slave-6k22 krb5kdc[13902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:26 dist-test-slave-6k22 krb5kdc[13902](info): set up 2 sockets
Aug 08 07:15:26 dist-test-slave-6k22 krb5kdc[13902](info): commencing operation
krb5kdc: starting...
W20260808 07:15:28.119249 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:28 dist-test-slave-6k22 krb5kdc[13902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:28.252677 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:28.335294 13918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:28.335544 13918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:28.335583 13918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:28.336794 13918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:28.336866 13918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:28.336890 13918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:28.336920 13918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:28.336944 13918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:28.338514 13918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:28.338806 13918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:28.339042 13918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:28.343978 13928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:28.344027 13926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:28.344033 13925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:28.344202 13918 server_base.cc:1034] running on GCE node
I20260808 07:15:28.344529 13918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:28.344817 13918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:28.345991 13918 hybrid_clock.cc:648] HybridClock initialized: now 1786173328345975 us; error 44 us; skew 500 ppm
Aug 08 07:15:28 dist-test-slave-6k22 krb5kdc[13902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173328, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:28.348526 13918 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:28.349004 13918 webserver.cc:460] Webserver started at http://127.0.0.1:41199/ using document root <none> and password file <none>
I20260808 07:15:28.349260 13918 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:28.349332 13918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:28.349480 13918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:28.350507 13918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "f63329c186d64ab0b1f30569933d783f"
format_stamp: "Formatted at 2026-08-08 07:15:28 on dist-test-slave-6k22"
server_key: "389c5e1030127a4fca85bc2b23169807"
server_key_iv: "44b85ec82b67bce2160605e7315da0b5"
server_key_version: "encryptionkey@0"
I20260808 07:15:28.350862 13918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "f63329c186d64ab0b1f30569933d783f"
format_stamp: "Formatted at 2026-08-08 07:15:28 on dist-test-slave-6k22"
server_key: "389c5e1030127a4fca85bc2b23169807"
server_key_iv: "44b85ec82b67bce2160605e7315da0b5"
server_key_version: "encryptionkey@0"
I20260808 07:15:28.352342 13918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:28.353106 13935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:28.353276 13918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:28.353384 13918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "f63329c186d64ab0b1f30569933d783f"
format_stamp: "Formatted at 2026-08-08 07:15:28 on dist-test-slave-6k22"
server_key: "389c5e1030127a4fca85bc2b23169807"
server_key_iv: "44b85ec82b67bce2160605e7315da0b5"
server_key_version: "encryptionkey@0"
I20260808 07:15:28.353458 13918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:28.370639 13918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:28.372865 13918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:28.373039 13918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:28.377513 13918 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38695
I20260808 07:15:28.377563 13993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38695 every 8 connection(s)
I20260808 07:15:28.377954 13918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:28.378230 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 13918
I20260808 07:15:28.378366 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:28.378593 22538 external_mini_cluster.cc:1428] Setting key 12b6743a1a385065e0af9601093cb22d
I20260808 07:15:28.379093 13994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:28.382385 13994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f: Bootstrap starting.
I20260808 07:15:28.383038 13994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:28.383363 13994 log.cc:824] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:28.384099 13994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f: No bootstrap required, opened a new log
Aug 08 07:15:28 dist-test-slave-6k22 krb5kdc[13902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:15:28.386011 13994 raft_consensus.cc:348] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63329c186d64ab0b1f30569933d783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260808 07:15:28.386170 13994 raft_consensus.cc:374] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:28.386240 13994 raft_consensus.cc:729] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f63329c186d64ab0b1f30569933d783f, State: Initialized, Role: FOLLOWER
I20260808 07:15:28.386411 13994 consensus_queue.cc:260] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63329c186d64ab0b1f30569933d783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260808 07:15:28.386514 13994 raft_consensus.cc:388] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:28.386550 13994 raft_consensus.cc:482] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:28.386623 13994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:28.387187 13994 raft_consensus.cc:504] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63329c186d64ab0b1f30569933d783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260808 07:15:28.387279 13994 leader_election.cc:302] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f63329c186d64ab0b1f30569933d783f; no voters:
I20260808 07:15:28.387455 13994 leader_election.cc:288] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:28.387508 13999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:28.387706 13999 raft_consensus.cc:686] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [term 1 LEADER]: Becoming Leader. State: Replica: f63329c186d64ab0b1f30569933d783f, State: Running, Role: LEADER
I20260808 07:15:28.387794 13994 sys_catalog.cc:564] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:28.387861 13999 consensus_queue.cc:237] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63329c186d64ab0b1f30569933d783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260808 07:15:28.389038 14000 sys_catalog.cc:455] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f63329c186d64ab0b1f30569933d783f. Latest consensus state: current_term: 1 leader_uuid: "f63329c186d64ab0b1f30569933d783f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63329c186d64ab0b1f30569933d783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260808 07:15:28.389125 14000 sys_catalog.cc:458] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:28.389317 14015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:28.389519 14001 sys_catalog.cc:455] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f63329c186d64ab0b1f30569933d783f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f63329c186d64ab0b1f30569933d783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260808 07:15:28.389636 14001 sys_catalog.cc:458] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:28.389982 14015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:28.390617 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:28.379151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:54103 (local address 127.0.0.1:38695)
0808 07:15:28.379342 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:28.379352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:28.379388 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:15:28.379935 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:28.379944 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:28.380011 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:28.380164 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:28.380789 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:28.382175 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:28.382958 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:28.383625 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:28.383750 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:28.386059 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:28.386064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:28.386064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:28.386082 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:28.388346 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:28.388860 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:28.388863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:28.388864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:28.388994 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:28.389301 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:28.389302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:28.389302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:28.389670 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:28.389815 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:28.390305 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 07:15:28.390408 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260808 07:15:28.391583 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:28.391649 14015 catalog_manager.cc:1347] Generated new cluster ID: bad46eeea12a49c4ab0b71765a59f886
I20260808 07:15:28.391703 14015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:28.401762 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:28.392953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:50907 (local address 127.0.0.1:38695)
0808 07:15:28.392994 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:28.392996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:28.395757 (+ 2761us) server_negotiation.cc:407] Connection header received
0808 07:15:28.395846 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:28.395847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:28.395857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:28.395906 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:28.396418 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:28.396897 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:28.397608 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:28.397847 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:28.397925 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:28.399123 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:28.399125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:28.399125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:28.399132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:28.400572 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:28.400959 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:28.400960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:28.400961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:28.401022 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:28.401243 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:28.401244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:28.401244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:28.401360 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:28.401473 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:28.401592 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 07:15:28.401655 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:28.407032 14015 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:28.407758 14015 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:28.413333 14015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f63329c186d64ab0b1f30569933d783f: Generated new TSK 0
I20260808 07:15:28.413504 14015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:28.418712 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 13918
W20260808 07:15:28.425272 13900 connection.cc:535] client connection to 127.0.0.1:38695 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38695): Connection reset by peer (error 104)
2026-08-08T07:15:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:28 dist-test-slave-6k22 krb5kdc[14026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:28 dist-test-slave-6k22 krb5kdc[14026](info): set up 2 sockets
Aug 08 07:15:28 dist-test-slave-6k22 krb5kdc[14026](info): commencing operation
krb5kdc: starting...
W20260808 07:15:29.555465 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:29 dist-test-slave-6k22 krb5kdc[14026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:29.687434 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:29.770809 14039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:29.771011 14039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:29.771034 14039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:29.772133 14039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:29.772176 14039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:29.772189 14039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:29.772202 14039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:29.772213 14039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:29.773723 14039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35745
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:29.773933 14039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:29.774103 14039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:29.779258 14044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:29.779309 14045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:29.779271 14047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:29.779489 14039 server_base.cc:1034] running on GCE node
I20260808 07:15:29.779919 14039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:29.780264 14039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:29.781450 14039 hybrid_clock.cc:648] HybridClock initialized: now 1786173329781429 us; error 31 us; skew 500 ppm
Aug 08 07:15:29 dist-test-slave-6k22 krb5kdc[14026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:29.784294 14039 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:29.784873 14039 webserver.cc:460] Webserver started at http://127.0.0.1:45357/ using document root <none> and password file <none>
I20260808 07:15:29.785125 14039 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:29.785199 14039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:29.785378 14039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:29.786549 14039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "78bd951934414dd29d82d38bc7e545c7"
format_stamp: "Formatted at 2026-08-08 07:15:29 on dist-test-slave-6k22"
server_key: "ab91ba0493acacd192cee4a005619309"
server_key_iv: "7ed7b9ac76b1774cd8980ea47dce5a3d"
server_key_version: "encryptionkey@0"
I20260808 07:15:29.786947 14039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "78bd951934414dd29d82d38bc7e545c7"
format_stamp: "Formatted at 2026-08-08 07:15:29 on dist-test-slave-6k22"
server_key: "ab91ba0493acacd192cee4a005619309"
server_key_iv: "7ed7b9ac76b1774cd8980ea47dce5a3d"
server_key_version: "encryptionkey@0"
I20260808 07:15:29.788527 14039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:29.789381 14054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:29.789561 14039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:29.789736 14039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "78bd951934414dd29d82d38bc7e545c7"
format_stamp: "Formatted at 2026-08-08 07:15:29 on dist-test-slave-6k22"
server_key: "ab91ba0493acacd192cee4a005619309"
server_key_iv: "7ed7b9ac76b1774cd8980ea47dce5a3d"
server_key_version: "encryptionkey@0"
I20260808 07:15:29.789826 14039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:29.813134 14039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:29.815150 14039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:29.815339 14039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:29.820051 14039 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35745
I20260808 07:15:29.820084 14110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35745 every 8 connection(s)
I20260808 07:15:29.820508 14039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:29.821403 14111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:29.823218 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 14039
I20260808 07:15:29.823374 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:29.823637 22538 external_mini_cluster.cc:1428] Setting key 81bb902eb98686fbb8e4ce8a2f4bb923
I20260808 07:15:29.823834 14111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7: Bootstrap starting.
I20260808 07:15:29.824635 14111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:29.824970 14111 log.cc:824] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:29.825835 14111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7: No bootstrap required, opened a new log
I20260808 07:15:29.828205 14111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bd951934414dd29d82d38bc7e545c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } }
I20260808 07:15:29.828387 14111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:29.828428 14111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bd951934414dd29d82d38bc7e545c7, State: Initialized, Role: FOLLOWER
I20260808 07:15:29.828558 14111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bd951934414dd29d82d38bc7e545c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } }
I20260808 07:15:29.828662 14111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:29.828692 14111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:29.828739 14111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:29.829547 14111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bd951934414dd29d82d38bc7e545c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } }
I20260808 07:15:29.829664 14111 leader_election.cc:302] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78bd951934414dd29d82d38bc7e545c7; no voters:
I20260808 07:15:29.829816 14111 leader_election.cc:288] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:29.829998 14119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:29.830108 14111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:29.830173 14119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [term 1 LEADER]: Becoming Leader. State: Replica: 78bd951934414dd29d82d38bc7e545c7, State: Running, Role: LEADER
I20260808 07:15:29.830318 14119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bd951934414dd29d82d38bc7e545c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } }
I20260808 07:15:29.830478 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:29.824159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:43295 (local address 127.0.0.1:35745)
0808 07:15:29.824367 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:29.824375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:29.824410 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:15:29.825067 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:29.825076 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:29.825147 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:29.825283 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:29.825566 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:29.826942 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:29.827776 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:29.828401 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:29.828509 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:29.829417 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:29.829421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:29.829421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:29.829637 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:29.829677 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:29.829692 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:29.829783 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:29.829988 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 07:15:29.830023 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260808 07:15:29.830724 14117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78bd951934414dd29d82d38bc7e545c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bd951934414dd29d82d38bc7e545c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } } }
I20260808 07:15:29.830828 14117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:29.830991 14120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78bd951934414dd29d82d38bc7e545c7. Latest consensus state: current_term: 1 leader_uuid: "78bd951934414dd29d82d38bc7e545c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bd951934414dd29d82d38bc7e545c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } } }
I20260808 07:15:29.831039 14120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:29.831459 14127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:29.831674 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:29.832351 14127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:29.834233 14127 catalog_manager.cc:1347] Generated new cluster ID: bc4b842f1abe4a0983c6a28006e00d91
I20260808 07:15:29.834295 14127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:29.836421 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:29.833299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:37651 (local address 127.0.0.1:35745)
0808 07:15:29.833340 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:29.833342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:29.833499 (+ 157us) server_negotiation.cc:407] Connection header received
0808 07:15:29.833575 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:29.833576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:29.833588 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:29.833651 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:29.833819 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:29.834405 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:29.835146 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:29.835379 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:29.835463 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:29.836169 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:29.836170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:29.836170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:29.836175 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:29.836188 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:29.836192 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:29.836245 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:29.836336 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:15:29.836346 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:15:29.849723 14127 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:29.850250 14127 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:29.852969 14127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78bd951934414dd29d82d38bc7e545c7: Generated new TSK 0
I20260808 07:15:29.853148 14127 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:29.855120 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 14039
W20260808 07:15:29.863498 14024 connection.cc:535] client connection to 127.0.0.1:35745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35745): Connection reset by peer (error 104)
2026-08-08T07:15:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:29 dist-test-slave-6k22 krb5kdc[14145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:29 dist-test-slave-6k22 krb5kdc[14145](info): set up 2 sockets
Aug 08 07:15:29 dist-test-slave-6k22 krb5kdc[14145](info): commencing operation
krb5kdc: starting...
W20260808 07:15:30.970537 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:31 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:31.102721 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:31.185053 14158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:31.185289 14158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:31.185319 14158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:31.186401 14158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:31.186437 14158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:31.186451 14158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:31.186465 14158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:31.186476 14158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:31.187922 14158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37371
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:31.188117 14158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:31.188289 14158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:31.193328 14163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:31.193379 14164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:31.193357 14158 server_base.cc:1034] running on GCE node
W20260808 07:15:31.193318 14166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:31.193827 14158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:31.194119 14158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:31.195282 14158 hybrid_clock.cc:648] HybridClock initialized: now 1786173331195269 us; error 47 us; skew 500 ppm
Aug 08 07:15:31 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173331, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:31.197845 14158 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:31.198319 14158 webserver.cc:460] Webserver started at http://127.0.0.1:42213/ using document root <none> and password file <none>
I20260808 07:15:31.198580 14158 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:31.198666 14158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:31.198817 14158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:31.199957 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "43a83cca4753489195e4aca187548751"
format_stamp: "Formatted at 2026-08-08 07:15:31 on dist-test-slave-6k22"
server_key: "6dd11cfad145d3568c0f48613535bd7e"
server_key_iv: "42f516fad705c45bbcfe81ddceed4943"
server_key_version: "encryptionkey@0"
I20260808 07:15:31.200352 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "43a83cca4753489195e4aca187548751"
format_stamp: "Formatted at 2026-08-08 07:15:31 on dist-test-slave-6k22"
server_key: "6dd11cfad145d3568c0f48613535bd7e"
server_key_iv: "42f516fad705c45bbcfe81ddceed4943"
server_key_version: "encryptionkey@0"
I20260808 07:15:31.201689 14158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:31.202483 14174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:31.202674 14158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:31.202800 14158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "43a83cca4753489195e4aca187548751"
format_stamp: "Formatted at 2026-08-08 07:15:31 on dist-test-slave-6k22"
server_key: "6dd11cfad145d3568c0f48613535bd7e"
server_key_iv: "42f516fad705c45bbcfe81ddceed4943"
server_key_version: "encryptionkey@0"
I20260808 07:15:31.202879 14158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:31.206579 14158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:31.209004 14158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:31.209172 14158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:31.213548 14158 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37371
I20260808 07:15:31.213596 14227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37371 every 8 connection(s)
I20260808 07:15:31.213968 14158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:31.214947 14228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:31.217348 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751: Bootstrap starting.
I20260808 07:15:31.217989 14228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:31.218060 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 14158
I20260808 07:15:31.218189 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:31.218381 14228 log.cc:824] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:31.218426 22538 external_mini_cluster.cc:1428] Setting key 47fb36d0fb6ff97ca625624b1f1f9754
I20260808 07:15:31.219292 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751: No bootstrap required, opened a new log
I20260808 07:15:31.221077 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:31.219018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:43593 (local address 127.0.0.1:37371)
0808 07:15:31.219177 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:31.219186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:31.219227 (+ 41us) server_negotiation.cc:407] Connection header received
0808 07:15:31.219774 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:31.219782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:31.219808 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:31.219925 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:31.220612 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:31.220615 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:31.220615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:31.220619 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:31.220656 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:31.220667 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:31.220744 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:31.220813 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:15:31.220880 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260808 07:15:31.222096 14228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a83cca4753489195e4aca187548751" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37371 } }
I20260808 07:15:31.222297 14228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:31.222265 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:31.222326 14228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a83cca4753489195e4aca187548751, State: Initialized, Role: FOLLOWER
I20260808 07:15:31.222461 14228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a83cca4753489195e4aca187548751" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37371 } }
I20260808 07:15:31.222554 14228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:31.222576 14228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:31.222641 14228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:31.223192 14228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a83cca4753489195e4aca187548751" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37371 } }
I20260808 07:15:31.223317 14228 leader_election.cc:302] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43a83cca4753489195e4aca187548751; no voters:
I20260808 07:15:31.223477 14228 leader_election.cc:288] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:31.223497 14238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:31.223764 14228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:31.223915 14238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [term 1 LEADER]: Becoming Leader. State: Replica: 43a83cca4753489195e4aca187548751, State: Running, Role: LEADER
I20260808 07:15:31.224016 14238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a83cca4753489195e4aca187548751" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37371 } }
I20260808 07:15:31.224387 14241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43a83cca4753489195e4aca187548751. Latest consensus state: current_term: 1 leader_uuid: "43a83cca4753489195e4aca187548751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a83cca4753489195e4aca187548751" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37371 } } }
I20260808 07:15:31.224493 14241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:31.224715 14248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:31.224848 14239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43a83cca4753489195e4aca187548751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a83cca4753489195e4aca187548751" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37371 } } }
I20260808 07:15:31.224989 14239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:31.225275 14248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:31.226189 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:31.223787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:50661 (local address 127.0.0.1:37371)
0808 07:15:31.223819 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:31.223821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:31.225272 (+ 1451us) server_negotiation.cc:407] Connection header received
0808 07:15:31.225338 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:31.225339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:31.225345 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:31.225401 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:31.225981 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:31.225982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:31.225983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:31.225986 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:31.225998 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:31.226001 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:31.226039 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:31.226097 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 07:15:31.226107 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:15:31.232229 14248 catalog_manager.cc:1347] Generated new cluster ID: 935fe94203d3465590544e02098c4112
I20260808 07:15:31.232293 14248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:31.252630 14248 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:31.252859 14248 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:31.256610 14248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43a83cca4753489195e4aca187548751: Generated new TSK 0
I20260808 07:15:31.256816 14248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:31.260844 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 14158
2026-08-08T07:15:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:31 dist-test-slave-6k22 krb5kdc[14261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:31 dist-test-slave-6k22 krb5kdc[14261](info): set up 2 sockets
Aug 08 07:15:31 dist-test-slave-6k22 krb5kdc[14261](info): commencing operation
krb5kdc: starting...
W20260808 07:15:32.383183 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:32 dist-test-slave-6k22 krb5kdc[14261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:32.511255 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45243
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:45243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:32.593942 14276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:32.594201 14276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:32.594242 14276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:32.595402 14276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:32.595471 14276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:32.595495 14276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:32.595517 14276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:32.595546 14276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:32.597047 14276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:45243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:45243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:32.597324 14276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:32.597570 14276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:32.602677 14284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:32.602677 14281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:32.602741 14282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:32.603058 14276 server_base.cc:1034] running on GCE node
I20260808 07:15:32.603324 14276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:32.603669 14276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:32.604817 14276 hybrid_clock.cc:648] HybridClock initialized: now 1786173332604802 us; error 31 us; skew 500 ppm
Aug 08 07:15:32 dist-test-slave-6k22 krb5kdc[14261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173332, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:32.612627 14276 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:32.613160 14276 webserver.cc:460] Webserver started at http://127.22.2.190:38903/ using document root <none> and password file <none>
I20260808 07:15:32.613427 14276 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:32.613499 14276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:32.613652 14276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:32.614759 14276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "702f7b306ec14cf181ed69b2ab4e3ed6"
format_stamp: "Formatted at 2026-08-08 07:15:32 on dist-test-slave-6k22"
server_key: "d01f3d5fa3d2d5fcdc9402383a7302bd"
server_key_iv: "16d3c0e230c8e80db14329521d8983ec"
server_key_version: "encryptionkey@0"
I20260808 07:15:32.615135 14276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "702f7b306ec14cf181ed69b2ab4e3ed6"
format_stamp: "Formatted at 2026-08-08 07:15:32 on dist-test-slave-6k22"
server_key: "d01f3d5fa3d2d5fcdc9402383a7302bd"
server_key_iv: "16d3c0e230c8e80db14329521d8983ec"
server_key_version: "encryptionkey@0"
I20260808 07:15:32.616590 14276 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:15:32.617326 14292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:32.617507 14276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:32.617615 14276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "702f7b306ec14cf181ed69b2ab4e3ed6"
format_stamp: "Formatted at 2026-08-08 07:15:32 on dist-test-slave-6k22"
server_key: "d01f3d5fa3d2d5fcdc9402383a7302bd"
server_key_iv: "16d3c0e230c8e80db14329521d8983ec"
server_key_version: "encryptionkey@0"
I20260808 07:15:32.617688 14276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:32.635267 14276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:32.637964 14276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:32.638155 14276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:32.642817 14276 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:45243
I20260808 07:15:32.642851 14349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:45243 every 8 connection(s)
I20260808 07:15:32.643250 14276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:32.644249 14350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:32.646490 14350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6: Bootstrap starting.
I20260808 07:15:32.646903 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 14276
I20260808 07:15:32.647037 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:32.647239 14350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:32.647281 22538 external_mini_cluster.cc:1428] Setting key fa35177589f8ffd6f6be281210592897
I20260808 07:15:32.647636 14350 log.cc:824] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:32.648473 14350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6: No bootstrap required, opened a new log
I20260808 07:15:32.650846 14350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702f7b306ec14cf181ed69b2ab4e3ed6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45243 } }
I20260808 07:15:32.651007 14350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:32.651043 14350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 702f7b306ec14cf181ed69b2ab4e3ed6, State: Initialized, Role: FOLLOWER
I20260808 07:15:32.651178 14350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702f7b306ec14cf181ed69b2ab4e3ed6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45243 } }
I20260808 07:15:32.651284 14350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:32.651322 14350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:32.651374 14350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:32.652040 14350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702f7b306ec14cf181ed69b2ab4e3ed6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45243 } }
I20260808 07:15:32.652141 14350 leader_election.cc:302] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 702f7b306ec14cf181ed69b2ab4e3ed6; no voters:
I20260808 07:15:32.652302 14350 leader_election.cc:288] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:32.652365 14356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:32.652554 14350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:32.652650 14356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [term 1 LEADER]: Becoming Leader. State: Replica: 702f7b306ec14cf181ed69b2ab4e3ed6, State: Running, Role: LEADER
I20260808 07:15:32.652772 14356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702f7b306ec14cf181ed69b2ab4e3ed6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45243 } }
I20260808 07:15:32.653154 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "702f7b306ec14cf181ed69b2ab4e3ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702f7b306ec14cf181ed69b2ab4e3ed6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45243 } } }
I20260808 07:15:32.653237 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:32.653483 14358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 702f7b306ec14cf181ed69b2ab4e3ed6. Latest consensus state: current_term: 1 leader_uuid: "702f7b306ec14cf181ed69b2ab4e3ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "702f7b306ec14cf181ed69b2ab4e3ed6" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 45243 } } }
I20260808 07:15:32.653497 14365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:32.653604 14358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:32.654168 14365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:15:32 dist-test-slave-6k22 krb5kdc[14261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:32.655740 14365 catalog_manager.cc:1347] Generated new cluster ID: aefc6e3c92144a27b0b7cfcba8df7992
I20260808 07:15:32.655794 14365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:32.659584 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:32.647866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46549 (local address 127.22.2.190:45243)
0808 07:15:32.648046 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:32.648056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:32.648091 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:15:32.648705 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:32.648714 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:32.648775 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:32.648924 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:32.649553 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:32.651039 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:32.651962 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:32.652590 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:32.653068 (+ 478us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:32.655829 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:32.655833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:32.655833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:32.655845 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:32.657812 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:32.658263 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:32.658267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:32.658267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:32.658416 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:32.658650 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:32.658651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:32.658651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:32.658961 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:32.659135 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:32.659234 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:15:32.659345 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260808 07:15:32.660825 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:32.665233 14365 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:32.665961 14365 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:32.669368 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:32.662285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53247 (local address 127.22.2.190:45243)
0808 07:15:32.662334 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:32.662336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:32.662349 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:15:32.662423 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:32.662424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:32.662434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:32.662483 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:32.663060 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:32.663836 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:32.664798 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:32.665109 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:32.665180 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:32.666588 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:32.666589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:32.666590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:32.666596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:32.668151 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:32.668573 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:32.668574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:32.668574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:32.668628 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:32.668853 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:32.668854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:32.668854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:32.668969 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:32.669064 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:32.669214 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:15:32.669278 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:15:32.671900 14365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 702f7b306ec14cf181ed69b2ab4e3ed6: Generated new TSK 0
I20260808 07:15:32.672093 14365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:32.676139 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 14276
W20260808 07:15:32.683840 14259 connection.cc:535] client connection to 127.22.2.190:45243 recv error: Network error: failed to read from TLS socket (remote: 127.22.2.190:45243): Connection reset by peer (error 104)
2026-08-08T07:15:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1413 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:32 dist-test-slave-6k22 krb5kdc[14383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:32 dist-test-slave-6k22 krb5kdc[14383](info): set up 2 sockets
Aug 08 07:15:32 dist-test-slave-6k22 krb5kdc[14383](info): commencing operation
krb5kdc: starting...
W20260808 07:15:33.798940 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:33 dist-test-slave-6k22 krb5kdc[14383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:33.920694 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36771
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:41481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:36771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:34.004436 14396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:34.004637 14396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:34.004666 14396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:34.005716 14396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:34.005748 14396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:34.005764 14396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:34.005775 14396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:34.005787 14396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:34.007195 14396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:36771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:34.007380 14396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:34.007539 14396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:34.013741 14405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:34.013703 14403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:34.013716 14402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:34.013927 14396 server_base.cc:1034] running on GCE node
I20260808 07:15:34.014150 14396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:34.014438 14396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:34.015623 14396 hybrid_clock.cc:648] HybridClock initialized: now 1786173334015662 us; error 90 us; skew 500 ppm
Aug 08 07:15:34 dist-test-slave-6k22 krb5kdc[14383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173334, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:34.018774 14396 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:34.019263 14396 webserver.cc:460] Webserver started at http://127.22.2.190:46107/ using document root <none> and password file <none>
I20260808 07:15:34.019444 14396 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:34.019483 14396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:34.019629 14396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:34.020653 14396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "0719e6aa9aa84c30b05993da6d4ac19c"
format_stamp: "Formatted at 2026-08-08 07:15:34 on dist-test-slave-6k22"
server_key: "24db7bce52096bc8341c79d4496dacb9"
server_key_iv: "8e7a56e0d12544b997f91b849576603c"
server_key_version: "encryptionkey@0"
I20260808 07:15:34.020957 14396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "0719e6aa9aa84c30b05993da6d4ac19c"
format_stamp: "Formatted at 2026-08-08 07:15:34 on dist-test-slave-6k22"
server_key: "24db7bce52096bc8341c79d4496dacb9"
server_key_iv: "8e7a56e0d12544b997f91b849576603c"
server_key_version: "encryptionkey@0"
I20260808 07:15:34.022243 14396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:15:34.022934 14412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:34.023110 14396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:34.023198 14396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "0719e6aa9aa84c30b05993da6d4ac19c"
format_stamp: "Formatted at 2026-08-08 07:15:34 on dist-test-slave-6k22"
server_key: "24db7bce52096bc8341c79d4496dacb9"
server_key_iv: "8e7a56e0d12544b997f91b849576603c"
server_key_version: "encryptionkey@0"
I20260808 07:15:34.023255 14396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:34.041489 14396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:34.043792 14396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:34.043900 14396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:34.047868 14396 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:36771
I20260808 07:15:34.047927 14469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:36771 every 8 connection(s)
I20260808 07:15:34.048272 14396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:34.049100 14471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:34.051343 14471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c: Bootstrap starting.
I20260808 07:15:34.051950 14471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:34.052237 14471 log.cc:824] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:34.052940 14471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c: No bootstrap required, opened a new log
I20260808 07:15:34.054566 14471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0719e6aa9aa84c30b05993da6d4ac19c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36771 } }
I20260808 07:15:34.054721 14471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:34.054768 14471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0719e6aa9aa84c30b05993da6d4ac19c, State: Initialized, Role: FOLLOWER
I20260808 07:15:34.054956 14471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0719e6aa9aa84c30b05993da6d4ac19c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36771 } }
I20260808 07:15:34.055059 14471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:34.055099 14471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:34.055151 14471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:34.055989 14471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0719e6aa9aa84c30b05993da6d4ac19c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36771 } }
I20260808 07:15:34.056104 14471 leader_election.cc:302] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0719e6aa9aa84c30b05993da6d4ac19c; no voters:
I20260808 07:15:34.056283 14471 leader_election.cc:288] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:34.056310 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 14396
I20260808 07:15:34.056335 14474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:34.056442 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:34.056466 14474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [term 1 LEADER]: Becoming Leader. State: Replica: 0719e6aa9aa84c30b05993da6d4ac19c, State: Running, Role: LEADER
I20260808 07:15:34.056551 14471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:34.056538 14474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0719e6aa9aa84c30b05993da6d4ac19c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36771 } }
I20260808 07:15:34.056646 22538 external_mini_cluster.cc:1428] Setting key 0ef151e4782341e21e3653fe63478693
I20260808 07:15:34.056839 14474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0719e6aa9aa84c30b05993da6d4ac19c. Latest consensus state: current_term: 1 leader_uuid: "0719e6aa9aa84c30b05993da6d4ac19c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0719e6aa9aa84c30b05993da6d4ac19c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36771 } } }
I20260808 07:15:34.057035 14474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:34.057325 14485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:34.057399 14477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0719e6aa9aa84c30b05993da6d4ac19c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0719e6aa9aa84c30b05993da6d4ac19c" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36771 } } }
I20260808 07:15:34.057499 14477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:34.057983 14485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:34.059578 14485 catalog_manager.cc:1347] Generated new cluster ID: 965f31165c4f41ddbff890cd7e3fd008
I20260808 07:15:34.059633 14485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:34.062688 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:34.057327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37139 (local address 127.22.2.190:36771)
0808 07:15:34.057506 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:34.057515 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:34.057898 (+ 383us) server_negotiation.cc:407] Connection header received
0808 07:15:34.058047 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:34.058054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:34.058136 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:34.058315 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:34.058541 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:34.059901 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:34.060709 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:34.061153 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:34.061282 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:34.061994 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:34.061998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:34.061998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:34.062191 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:34.062221 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:34.062232 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:34.062324 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:34.062405 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:15:34.062530 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260808 07:15:34.063623 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:34.070873 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:34.067916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45999 (local address 127.22.2.190:36771)
0808 07:15:34.067952 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:34.067954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:34.068143 (+ 189us) server_negotiation.cc:407] Connection header received
0808 07:15:34.068225 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:34.068226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:34.068250 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:34.068307 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:34.068447 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:34.068934 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:34.069629 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:34.069845 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:34.069918 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:34.070592 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:34.070594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:34.070594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:34.070598 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:34.070612 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:34.070616 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:34.070699 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:34.070762 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:15:34.070774 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:34.072414 14485 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:34.072953 14485 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:34.080439 14485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0719e6aa9aa84c30b05993da6d4ac19c: Generated new TSK 0
I20260808 07:15:34.080617 14485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:34.085707 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 14396
W20260808 07:15:34.094205 14381 connection.cc:535] client connection to 127.22.2.190:36771 recv error: Network error: failed to read from TLS socket (remote: 127.22.2.190:36771): Connection reset by peer (error 104)
2026-08-08T07:15:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:34 dist-test-slave-6k22 krb5kdc[14504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:34 dist-test-slave-6k22 krb5kdc[14504](info): set up 2 sockets
Aug 08 07:15:34 dist-test-slave-6k22 krb5kdc[14504](info): commencing operation
krb5kdc: starting...
W20260808 07:15:35.196286 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:35 dist-test-slave-6k22 krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:35.322525 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46001
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:41597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:46001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:35.403628 14518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:35.403874 14518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:35.403913 14518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:35.405048 14518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:35.405118 14518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:35.405146 14518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:35.405179 14518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:35.405200 14518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:35.406656 14518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:41597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:46001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:35.406934 14518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:35.407157 14518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:35.412043 14526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:35.412050 14523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:35.412372 14518 server_base.cc:1034] running on GCE node
W20260808 07:15:35.412081 14524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:35.412662 14518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:35.412912 14518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:35.414076 14518 hybrid_clock.cc:648] HybridClock initialized: now 1786173335414061 us; error 42 us; skew 500 ppm
Aug 08 07:15:35 dist-test-slave-6k22 krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173335, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:35.416457 14518 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:35.416929 14518 webserver.cc:460] Webserver started at http://127.22.2.190:43497/ using document root <none> and password file <none>
I20260808 07:15:35.417158 14518 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:35.417227 14518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:35.417380 14518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:35.418427 14518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "be4e6365ce204cb4bbab6b8f1ed4148b"
format_stamp: "Formatted at 2026-08-08 07:15:35 on dist-test-slave-6k22"
server_key: "b98f6e322387edee294a2ab52034c216"
server_key_iv: "7d1dc6dd2a44a2ec3cea2c934661e3b9"
server_key_version: "encryptionkey@0"
I20260808 07:15:35.418793 14518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "be4e6365ce204cb4bbab6b8f1ed4148b"
format_stamp: "Formatted at 2026-08-08 07:15:35 on dist-test-slave-6k22"
server_key: "b98f6e322387edee294a2ab52034c216"
server_key_iv: "7d1dc6dd2a44a2ec3cea2c934661e3b9"
server_key_version: "encryptionkey@0"
I20260808 07:15:35.420219 14518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:15:35.420933 14534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:35.421104 14518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:35.421219 14518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "be4e6365ce204cb4bbab6b8f1ed4148b"
format_stamp: "Formatted at 2026-08-08 07:15:35 on dist-test-slave-6k22"
server_key: "b98f6e322387edee294a2ab52034c216"
server_key_iv: "7d1dc6dd2a44a2ec3cea2c934661e3b9"
server_key_version: "encryptionkey@0"
I20260808 07:15:35.421310 14518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:35.424615 14518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:35.426501 14518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:35.426723 14518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:35.431025 14518 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:46001
I20260808 07:15:35.431064 14592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:46001 every 8 connection(s)
I20260808 07:15:35.431402 14518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:35.432468 14593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:35.434938 14593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b: Bootstrap starting.
I20260808 07:15:35.435632 14593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:35.435943 14593 log.cc:824] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:35.436721 14593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b: No bootstrap required, opened a new log
I20260808 07:15:35.437424 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 14518
I20260808 07:15:35.437562 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:35.437748 22538 external_mini_cluster.cc:1428] Setting key 93a5441809adc7c40360009f0a1ee83c
I20260808 07:15:35.439432 14593 raft_consensus.cc:348] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4e6365ce204cb4bbab6b8f1ed4148b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46001 } }
I20260808 07:15:35.439797 14593 raft_consensus.cc:374] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:35.439859 14593 raft_consensus.cc:729] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be4e6365ce204cb4bbab6b8f1ed4148b, State: Initialized, Role: FOLLOWER
I20260808 07:15:35.439998 14593 consensus_queue.cc:260] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4e6365ce204cb4bbab6b8f1ed4148b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46001 } }
I20260808 07:15:35.440135 14593 raft_consensus.cc:388] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:35.440173 14593 raft_consensus.cc:482] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:35.440237 14593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:35.440778 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:35.438289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47013 (local address 127.22.2.190:46001)
0808 07:15:35.438469 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:35.438476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:35.438502 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:15:35.439321 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:35.439328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:35.439362 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:35.439475 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:35.440282 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:35.440285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:35.440286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:35.440291 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:35.440335 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:35.440348 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:35.440415 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:35.440518 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:15:35.440584 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260808 07:15:35.440964 14593 raft_consensus.cc:504] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4e6365ce204cb4bbab6b8f1ed4148b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46001 } }
I20260808 07:15:35.441077 14593 leader_election.cc:302] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be4e6365ce204cb4bbab6b8f1ed4148b; no voters:
I20260808 07:15:35.441246 14593 leader_election.cc:288] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:35.441300 14598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:35.441457 14598 raft_consensus.cc:686] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [term 1 LEADER]: Becoming Leader. State: Replica: be4e6365ce204cb4bbab6b8f1ed4148b, State: Running, Role: LEADER
I20260808 07:15:35.441534 14598 consensus_queue.cc:237] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4e6365ce204cb4bbab6b8f1ed4148b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46001 } }
I20260808 07:15:35.441545 14593 sys_catalog.cc:564] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:35.442081 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:35.442122 14600 sys_catalog.cc:455] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [sys.catalog]: SysCatalogTable state changed. Reason: New leader be4e6365ce204cb4bbab6b8f1ed4148b. Latest consensus state: current_term: 1 leader_uuid: "be4e6365ce204cb4bbab6b8f1ed4148b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4e6365ce204cb4bbab6b8f1ed4148b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46001 } } }
I20260808 07:15:35.442217 14600 sys_catalog.cc:458] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:35.442373 14601 sys_catalog.cc:455] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be4e6365ce204cb4bbab6b8f1ed4148b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be4e6365ce204cb4bbab6b8f1ed4148b" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46001 } } }
W20260808 07:15:35.443706 14622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:35.444088 14622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:35.444063 14601 sys_catalog.cc:458] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:35.444221 14606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:35.444401 14606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:35.446216 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:35.443969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48465 (local address 127.22.2.190:46001)
0808 07:15:35.444006 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:35.444008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:35.444016 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:35.445071 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:35.445072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:35.445077 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:35.445112 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:35.445951 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:35.445952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:35.445952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:35.445956 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:35.445968 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:35.445971 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:35.446064 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:35.446128 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:15:35.446142 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:35.446360 14606 catalog_manager.cc:1347] Generated new cluster ID: 1f7962ac21ac4ec4977198fedf1f5551
I20260808 07:15:35.446404 14606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:35.459743 14606 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:35.459928 14606 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:35.463615 14606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be4e6365ce204cb4bbab6b8f1ed4148b: Generated new TSK 0
I20260808 07:15:35.463802 14606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:35.473220 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 14518
2026-08-08T07:15:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1388 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:35 dist-test-slave-6k22 krb5kdc[14631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:35 dist-test-slave-6k22 krb5kdc[14631](info): set up 2 sockets
Aug 08 07:15:35 dist-test-slave-6k22 krb5kdc[14631](info): commencing operation
krb5kdc: starting...
W20260808 07:15:36.594208 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:36 dist-test-slave-6k22 krb5kdc[14631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:36.719112 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46347
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:46347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:36.799386 14644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:36.799681 14644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:36.799724 14644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:36.800895 14644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:36.800964 14644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:36.800992 14644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:36.801023 14644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:36.801047 14644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:36.802515 14644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:46347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:46347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:36.802795 14644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:36.803018 14644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:36.807996 14650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:36.807999 14652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:36.808017 14649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:36.808250 14644 server_base.cc:1034] running on GCE node
I20260808 07:15:36.808456 14644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:36.808676 14644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:36.809811 14644 hybrid_clock.cc:648] HybridClock initialized: now 1786173336809800 us; error 31 us; skew 500 ppm
Aug 08 07:15:36 dist-test-slave-6k22 krb5kdc[14631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173336, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:36.812352 14644 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:36.812808 14644 webserver.cc:460] Webserver started at http://127.22.2.190:34137/ using document root <none> and password file <none>
I20260808 07:15:36.813046 14644 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:36.813115 14644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:36.813275 14644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:36.814314 14644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "a7a02ebc7c4642e2b042908f31a9e33d"
format_stamp: "Formatted at 2026-08-08 07:15:36 on dist-test-slave-6k22"
server_key: "6b9b62513638cb19c7d607fa966fbb21"
server_key_iv: "690394600310c9505ec031a23377765d"
server_key_version: "encryptionkey@0"
I20260808 07:15:36.814682 14644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "a7a02ebc7c4642e2b042908f31a9e33d"
format_stamp: "Formatted at 2026-08-08 07:15:36 on dist-test-slave-6k22"
server_key: "6b9b62513638cb19c7d607fa966fbb21"
server_key_iv: "690394600310c9505ec031a23377765d"
server_key_version: "encryptionkey@0"
I20260808 07:15:36.816107 14644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:36.816810 14660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:36.816985 14644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:36.817092 14644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "a7a02ebc7c4642e2b042908f31a9e33d"
format_stamp: "Formatted at 2026-08-08 07:15:36 on dist-test-slave-6k22"
server_key: "6b9b62513638cb19c7d607fa966fbb21"
server_key_iv: "690394600310c9505ec031a23377765d"
server_key_version: "encryptionkey@0"
I20260808 07:15:36.817165 14644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:36.830731 14644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:36.833082 14644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:36.833254 14644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:36.837507 14644 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:46347
I20260808 07:15:36.837543 14718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:46347 every 8 connection(s)
I20260808 07:15:36.837882 14644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:36.838814 14720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:36.841285 14720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d: Bootstrap starting.
I20260808 07:15:36.841830 14720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:36.842094 14720 log.cc:824] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:36.842830 14720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d: No bootstrap required, opened a new log
I20260808 07:15:36.844071 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 14644
I20260808 07:15:36.844214 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:36.844429 22538 external_mini_cluster.cc:1428] Setting key 41b1487b1c12e133edfc2dd0bc45910b
I20260808 07:15:36.844964 14720 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a02ebc7c4642e2b042908f31a9e33d" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46347 } }
I20260808 07:15:36.845211 14720 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:36.845276 14720 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7a02ebc7c4642e2b042908f31a9e33d, State: Initialized, Role: FOLLOWER
I20260808 07:15:36.845427 14720 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a02ebc7c4642e2b042908f31a9e33d" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46347 } }
I20260808 07:15:36.845510 14720 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:36.845535 14720 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:36.845609 14720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:36.846498 14720 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a02ebc7c4642e2b042908f31a9e33d" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46347 } }
I20260808 07:15:36.846628 14720 leader_election.cc:302] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7a02ebc7c4642e2b042908f31a9e33d; no voters:
I20260808 07:15:36.846815 14720 leader_election.cc:288] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:36.846853 14725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:36.846969 14725 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [term 1 LEADER]: Becoming Leader. State: Replica: a7a02ebc7c4642e2b042908f31a9e33d, State: Running, Role: LEADER
I20260808 07:15:36.847090 14720 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:36.847119 14725 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a02ebc7c4642e2b042908f31a9e33d" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46347 } }
I20260808 07:15:36.847589 14725 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7a02ebc7c4642e2b042908f31a9e33d. Latest consensus state: current_term: 1 leader_uuid: "a7a02ebc7c4642e2b042908f31a9e33d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a02ebc7c4642e2b042908f31a9e33d" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46347 } } }
I20260808 07:15:36.847958 14725 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:36.848470 14734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:36.847788 14726 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7a02ebc7c4642e2b042908f31a9e33d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a02ebc7c4642e2b042908f31a9e33d" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 46347 } } }
I20260808 07:15:36.848659 14726 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:36.849095 14734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:36.851321 14734 catalog_manager.cc:1347] Generated new cluster ID: dead39f5abe14a4f9dfe91def4ae7a28
I20260808 07:15:36.851382 14734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:36 dist-test-slave-6k22 krb5kdc[14631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:36.855512 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:36.845010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39953 (local address 127.22.2.190:46347)
0808 07:15:36.845217 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:36.845226 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:36.845260 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:15:36.845379 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:36.845385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:36.845450 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:36.845609 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:36.846362 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:36.847532 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:36.848717 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:36.849341 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:36.849448 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:36.852334 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:36.852337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:36.852338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:36.852349 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:36.853976 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:36.854338 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:36.854341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:36.854341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:36.854441 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:36.854667 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:36.854668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:36.854668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:36.854914 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:36.855095 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:36.855162 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:15:36.855314 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260808 07:15:36.856385 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:36.863754 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:36.857726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51845 (local address 127.22.2.190:46347)
0808 07:15:36.857764 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:36.857765 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:36.857887 (+ 122us) server_negotiation.cc:407] Connection header received
0808 07:15:36.857959 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:36.857960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:36.857970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:36.858013 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:36.858539 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:36.859005 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:36.859764 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:36.859992 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:36.860069 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:36.861199 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:36.861200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:36.861201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:36.861207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:36.862577 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:36.862940 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:36.862941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:36.862941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:36.863033 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:36.863237 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:36.863238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:36.863239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:36.863350 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:36.863450 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:36.863572 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 07:15:36.863646 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:36.873023 14734 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:36.873696 14734 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:36.879590 14734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7a02ebc7c4642e2b042908f31a9e33d: Generated new TSK 0
I20260808 07:15:36.879771 14734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:36.889359 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 14644
2026-08-08T07:15:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:36 dist-test-slave-6k22 krb5kdc[14751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:36 dist-test-slave-6k22 krb5kdc[14751](info): set up 2 sockets
Aug 08 07:15:36 dist-test-slave-6k22 krb5kdc[14751](info): commencing operation
krb5kdc: starting...
W20260808 07:15:37.974440 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:38 dist-test-slave-6k22 krb5kdc[14751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:38.119845 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36745
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:36745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:38.207383 14767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:38.207607 14767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:38.207630 14767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:38.208714 14767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:38.208746 14767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:38.208760 14767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:38.208773 14767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:38.208786 14767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:38.210179 14767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:36745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:36745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:38.210448 14767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:38.210659 14767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:38.215508 14777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:38.215493 14775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:38.215551 14767 server_base.cc:1034] running on GCE node
W20260808 07:15:38.215474 14774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:38.215869 14767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:38.216159 14767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:38.217319 14767 hybrid_clock.cc:648] HybridClock initialized: now 1786173338217308 us; error 46 us; skew 500 ppm
Aug 08 07:15:38 dist-test-slave-6k22 krb5kdc[14751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173338, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:38.219795 14767 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:38.220263 14767 webserver.cc:460] Webserver started at http://127.22.2.190:43107/ using document root <none> and password file <none>
I20260808 07:15:38.220489 14767 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:38.220556 14767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:38.220717 14767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:38.221745 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "c27a68493d0e456aa554672a9284c485"
format_stamp: "Formatted at 2026-08-08 07:15:38 on dist-test-slave-6k22"
server_key: "38dde535194480edb2f62181fbf0b26f"
server_key_iv: "9a5d0a3e0108bbd6a419909e04c8a8c6"
server_key_version: "encryptionkey@0"
I20260808 07:15:38.222164 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "c27a68493d0e456aa554672a9284c485"
format_stamp: "Formatted at 2026-08-08 07:15:38 on dist-test-slave-6k22"
server_key: "38dde535194480edb2f62181fbf0b26f"
server_key_iv: "9a5d0a3e0108bbd6a419909e04c8a8c6"
server_key_version: "encryptionkey@0"
I20260808 07:15:38.223482 14767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:15:38.224242 14784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:38.224444 14767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:38.224535 14767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "c27a68493d0e456aa554672a9284c485"
format_stamp: "Formatted at 2026-08-08 07:15:38 on dist-test-slave-6k22"
server_key: "38dde535194480edb2f62181fbf0b26f"
server_key_iv: "9a5d0a3e0108bbd6a419909e04c8a8c6"
server_key_version: "encryptionkey@0"
I20260808 07:15:38.224587 14767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:38.235982 14767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:38.238142 14767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:38.238253 14767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:38.242255 14767 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:36745
I20260808 07:15:38.242307 14839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:36745 every 8 connection(s)
I20260808 07:15:38.242659 14767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:38.243625 14840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:38.244786 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 14767
I20260808 07:15:38.244908 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:38.245112 22538 external_mini_cluster.cc:1428] Setting key 12f7cf1f336eaac798dc0babd1da9845
I20260808 07:15:38.246456 14840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485: Bootstrap starting.
I20260808 07:15:38.247161 14840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:38.247476 14840 log.cc:824] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:38.248247 14840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485: No bootstrap required, opened a new log
I20260808 07:15:38.250741 14840 raft_consensus.cc:348] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27a68493d0e456aa554672a9284c485" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36745 } }
I20260808 07:15:38.250932 14840 raft_consensus.cc:374] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:38.251013 14840 raft_consensus.cc:729] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27a68493d0e456aa554672a9284c485, State: Initialized, Role: FOLLOWER
I20260808 07:15:38.251200 14840 consensus_queue.cc:260] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27a68493d0e456aa554672a9284c485" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36745 } }
I20260808 07:15:38.251305 14840 raft_consensus.cc:388] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:38.251356 14840 raft_consensus.cc:482] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:38.251403 14840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:38.251408 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:38.245640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59475 (local address 127.22.2.190:36745)
0808 07:15:38.245851 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:38.245861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:38.245897 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:15:38.246428 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:38.246436 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:38.246497 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:38.246640 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:38.247002 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:38.248032 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:38.249152 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:38.249564 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:38.249641 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:38.250476 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:38.250480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:38.250481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:38.250724 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:38.250805 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:38.250820 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:38.250902 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:38.251086 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 07:15:38.251160 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260808 07:15:38.252080 14840 raft_consensus.cc:504] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27a68493d0e456aa554672a9284c485" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36745 } }
I20260808 07:15:38.252199 14840 leader_election.cc:302] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c27a68493d0e456aa554672a9284c485; no voters:
I20260808 07:15:38.252369 14840 leader_election.cc:288] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:38.252398 14845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:38.252491 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:38.252585 14845 raft_consensus.cc:686] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [term 1 LEADER]: Becoming Leader. State: Replica: c27a68493d0e456aa554672a9284c485, State: Running, Role: LEADER
I20260808 07:15:38.252700 14845 consensus_queue.cc:237] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27a68493d0e456aa554672a9284c485" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36745 } }
I20260808 07:15:38.252609 14840 sys_catalog.cc:564] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:38.253026 14846 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c27a68493d0e456aa554672a9284c485. Latest consensus state: current_term: 1 leader_uuid: "c27a68493d0e456aa554672a9284c485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27a68493d0e456aa554672a9284c485" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36745 } } }
I20260808 07:15:38.253108 14846 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:38.253294 14847 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c27a68493d0e456aa554672a9284c485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27a68493d0e456aa554672a9284c485" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 36745 } } }
I20260808 07:15:38.253594 14847 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:38.254050 14849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:38.254830 14849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:38.258944 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:38.253989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50181 (local address 127.22.2.190:36745)
0808 07:15:38.254023 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:38.254025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:38.254494 (+ 469us) server_negotiation.cc:407] Connection header received
0808 07:15:38.254557 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:38.254558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:38.254568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:38.254622 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:38.254772 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:38.255414 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:38.256339 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:38.256619 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:38.256712 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:38.258658 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:38.258659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:38.258660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:38.258664 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:38.258678 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:38.258682 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:38.258709 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:38.258820 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 07:15:38.258844 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:38.259474 14849 catalog_manager.cc:1347] Generated new cluster ID: d41a4950b12b45978fde0f268617072c
I20260808 07:15:38.259538 14849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:38.267282 14849 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:38.267884 14849 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:38.280906 14849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c27a68493d0e456aa554672a9284c485: Generated new TSK 0
I20260808 07:15:38.281093 14849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:38.287743 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 14767
2026-08-08T07:15:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:38 dist-test-slave-6k22 krb5kdc[14872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:38 dist-test-slave-6k22 krb5kdc[14872](info): set up 2 sockets
Aug 08 07:15:38 dist-test-slave-6k22 krb5kdc[14872](info): commencing operation
krb5kdc: starting...
W20260808 07:15:39.406865 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:39 dist-test-slave-6k22 krb5kdc[14872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:39.530977 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37411
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:37411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:39.615370 14885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:39.615700 14885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:39.615731 14885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:39.616906 14885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:39.616951 14885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:39.616967 14885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:39.617024 14885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:39.617051 14885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:39.618618 14885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:37411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:37411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:39.618875 14885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:39.619124 14885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:15:39.624294 14885 server_base.cc:1034] running on GCE node
W20260808 07:15:39.624200 14890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:39.624186 14891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:39.624186 14893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:39.624756 14885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:39.624997 14885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:39.626138 14885 hybrid_clock.cc:648] HybridClock initialized: now 1786173339626117 us; error 36 us; skew 500 ppm
Aug 08 07:15:39 dist-test-slave-6k22 krb5kdc[14872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173339, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:39.628696 14885 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:39.629069 14885 webserver.cc:460] Webserver started at http://127.22.2.190:40231/ using document root <none> and password file <none>
I20260808 07:15:39.629262 14885 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:39.629302 14885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:39.629398 14885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:39.630369 14885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "49553ee9e6674420a5c62e1768d9cde3"
format_stamp: "Formatted at 2026-08-08 07:15:39 on dist-test-slave-6k22"
server_key: "2d2d2b6063b2ee895f16bddd90614f1c"
server_key_iv: "3c141526c925142a04b06e362aed4606"
server_key_version: "encryptionkey@0"
I20260808 07:15:39.630663 14885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "49553ee9e6674420a5c62e1768d9cde3"
format_stamp: "Formatted at 2026-08-08 07:15:39 on dist-test-slave-6k22"
server_key: "2d2d2b6063b2ee895f16bddd90614f1c"
server_key_iv: "3c141526c925142a04b06e362aed4606"
server_key_version: "encryptionkey@0"
I20260808 07:15:39.631955 14885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:39.632656 14902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:39.632827 14885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:39.632936 14885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "49553ee9e6674420a5c62e1768d9cde3"
format_stamp: "Formatted at 2026-08-08 07:15:39 on dist-test-slave-6k22"
server_key: "2d2d2b6063b2ee895f16bddd90614f1c"
server_key_iv: "3c141526c925142a04b06e362aed4606"
server_key_version: "encryptionkey@0"
I20260808 07:15:39.633013 14885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:39.636307 14885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:39.638556 14885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:39.638712 14885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:39.642825 14885 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:37411
I20260808 07:15:39.642859 14962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:37411 every 8 connection(s)
I20260808 07:15:39.643177 14885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:39.644232 14963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:39.646070 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 14885
I20260808 07:15:39.646183 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:39.646412 22538 external_mini_cluster.cc:1428] Setting key 0707014a4998c4a3753c97f7ba4b6536
I20260808 07:15:39.646813 14963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3: Bootstrap starting.
I20260808 07:15:39.647451 14963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:39.647823 14963 log.cc:824] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:39.648679 14963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3: No bootstrap required, opened a new log
I20260808 07:15:39.648929 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:39.646939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50605 (local address 127.22.2.190:37411)
0808 07:15:39.647117 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:39.647126 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:39.647151 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:15:39.647655 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:39.647661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:39.647689 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:39.647806 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:39.648519 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:39.648522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:39.648523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:39.648526 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:39.648560 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:39.648570 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:39.648640 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:39.648711 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:15:39.648768 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260808 07:15:39.649984 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:39.651077 14963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49553ee9e6674420a5c62e1768d9cde3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37411 } }
I20260808 07:15:39.651250 14963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:39.651355 14963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49553ee9e6674420a5c62e1768d9cde3, State: Initialized, Role: FOLLOWER
I20260808 07:15:39.651499 14963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49553ee9e6674420a5c62e1768d9cde3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37411 } }
I20260808 07:15:39.651631 14963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:39.651719 14963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:39.651800 14963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:39.652442 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:39.651448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39675 (local address 127.22.2.190:37411)
0808 07:15:39.651482 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:39.651484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:39.651494 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:39.651592 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:39.651593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:39.651597 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:39.651666 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:39.652221 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:39.652222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:39.652222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:39.652226 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:39.652239 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:39.652243 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:39.652313 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:39.652350 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:15:39.652363 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:39.652442 14963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49553ee9e6674420a5c62e1768d9cde3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37411 } }
I20260808 07:15:39.652570 14963 leader_election.cc:302] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49553ee9e6674420a5c62e1768d9cde3; no voters:
I20260808 07:15:39.652727 14963 leader_election.cc:288] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:39.652766 14973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:39.652880 14973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [term 1 LEADER]: Becoming Leader. State: Replica: 49553ee9e6674420a5c62e1768d9cde3, State: Running, Role: LEADER
I20260808 07:15:39.652985 14963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:39.653014 14973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49553ee9e6674420a5c62e1768d9cde3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37411 } }
I20260808 07:15:39.653339 14973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49553ee9e6674420a5c62e1768d9cde3. Latest consensus state: current_term: 1 leader_uuid: "49553ee9e6674420a5c62e1768d9cde3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49553ee9e6674420a5c62e1768d9cde3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37411 } } }
I20260808 07:15:39.653352 14974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49553ee9e6674420a5c62e1768d9cde3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49553ee9e6674420a5c62e1768d9cde3" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 37411 } } }
I20260808 07:15:39.653497 14973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:39.653580 14974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3 [sys.catalog]: This master's current role is: LEADER
W20260808 07:15:39.654729 14990 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:39.654794 14990 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:39.654853 14983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:39.655007 14983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:39.656424 14983 catalog_manager.cc:1347] Generated new cluster ID: 873796ce34ef4eef86e695d318806143
I20260808 07:15:39.656476 14983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:39.683866 14983 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:39.684073 14983 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:39.687986 14983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49553ee9e6674420a5c62e1768d9cde3: Generated new TSK 0
I20260808 07:15:39.688189 14983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:39.690949 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 14885
2026-08-08T07:15:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1403 ms)
[----------] 18 tests from AuthTokenIssuingTest (25449 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:39 dist-test-slave-6k22 krb5kdc[14995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:39 dist-test-slave-6k22 krb5kdc[14995](info): set up 2 sockets
Aug 08 07:15:39 dist-test-slave-6k22 krb5kdc[14995](info): commencing operation
krb5kdc: starting...
W20260808 07:15:40.795315 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:40 dist-test-slave-6k22 krb5kdc[14995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:40.918641 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44351
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:44351,127.22.2.189:37203,127.22.2.188:36163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:41.004966 15010 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:41.005190 15010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:41.005209 15010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:41.005265 15010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:41.006389 15010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:41.006429 15010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:41.006443 15010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:41.006486 15010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:41.006515 15010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:41.008184 15010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:44351,127.22.2.189:37203,127.22.2.188:36163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:44351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:41.008452 15010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:41.008678 15010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:41.013818 15019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:41.014008 15010 server_base.cc:1034] running on GCE node
W20260808 07:15:41.013854 15016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:41.013814 15017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:41.014406 15010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:41.014703 15010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:41.015863 15010 hybrid_clock.cc:648] HybridClock initialized: now 1786173341015850 us; error 43 us; skew 500 ppm
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:41.018392 15010 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:41.018774 15010 webserver.cc:460] Webserver started at http://127.22.2.190:45611/ using document root <none> and password file <none>
I20260808 07:15:41.018952 15010 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:41.018992 15010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:41.019080 15010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:41.020165 15010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "ea76f38873644680bd05f33ef35b02eb"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "fa9f7dea8ba68c09469f507356fb92a4"
server_key_iv: "7feb95545703e84c0f1a1b92edcac640"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.020483 15010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "ea76f38873644680bd05f33ef35b02eb"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "fa9f7dea8ba68c09469f507356fb92a4"
server_key_iv: "7feb95545703e84c0f1a1b92edcac640"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.021855 15010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:41.022523 15026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:41.022745 15010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:41.022838 15010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "ea76f38873644680bd05f33ef35b02eb"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "fa9f7dea8ba68c09469f507356fb92a4"
server_key_iv: "7feb95545703e84c0f1a1b92edcac640"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.022893 15010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:41.054865 15010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:41.056869 15010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:41.056983 15010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:41.060959 15010 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:44351
I20260808 07:15:41.061020 15086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:44351 every 8 connection(s)
I20260808 07:15:41.061391 15010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:41.062418 15087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:41.064074 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 15010
I20260808 07:15:41.064167 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:41.064045 15087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:41.064419 22538 external_mini_cluster.cc:1428] Setting key d0b557c0a18ca6236cb57a597cd1b88e
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.189@KRBTEST.COM; defaulting to no policy
I20260808 07:15:41.073993 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.064894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60330 (local address 127.22.2.190:44351)
0808 07:15:41.065125 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.065129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.065190 (+ 61us) server_negotiation.cc:407] Connection header received
0808 07:15:41.065748 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.065750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.065770 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.065888 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.066449 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.067261 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.068215 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.068548 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.068665 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.070754 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.070761 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.070761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.070768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.072387 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.072768 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.072771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.072771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.072858 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.073090 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.073091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.073091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.073294 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.073390 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.073571 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.073714 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
I20260808 07:15:41.073993 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.064874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44351 (local address 127.0.0.1:60330)
0808 07:15:41.065055 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.065099 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.065591 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.065896 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.065899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.066184 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.066329 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.066342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.067363 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.067363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.068112 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.068114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.068664 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.068665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.068731 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.069275 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.069283 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.070663 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.072477 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.072484 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.072486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.072676 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.072930 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.072930 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.072931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.073017 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.073369 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.073374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.073434 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.073546 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.073779 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":95,"thread_start_us":101,"threads_started":1}
I20260808 07:15:41.074759 15087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.189@KRBTEST.COM" created.
I20260808 07:15:41.075217 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.075047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43140)
0808 07:15:41.075121 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.075189 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 07:15:41.075398 15029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
I20260808 07:15:41.075913 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.075787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43148)
0808 07:15:41.075831 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.075861 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:15:41.076038 15087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:37203: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
I20260808 07:15:41.089944 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:37203
--webserver_interface=127.22.2.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:44351,127.22.2.189:37203,127.22.2.188:36163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:15:41.125252 15087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } attempt: 1
I20260808 07:15:41.125732 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.125579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43150)
0808 07:15:41.125640 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.125680 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:15:41.126134 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.126038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43160)
0808 07:15:41.126082 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.126109 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 07:15:41.126267 15087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:37203: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
W20260808 07:15:41.171342 15095 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:41.171614 15095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:41.171646 15095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:41.171687 15095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:41.172935 15095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:41.172995 15095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:41.173015 15095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:41.173054 15095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:41.173086 15095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:41.174588 15095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:44351,127.22.2.189:37203,127.22.2.188:36163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:37203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.2.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:41.174860 15095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:41.175113 15095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:41.179951 15101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:41.179960 15100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:41.180060 15095 server_base.cc:1034] running on GCE node
W20260808 07:15:41.179965 15103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:41.180371 15095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:41.180660 15095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:41.181823 15095 hybrid_clock.cc:648] HybridClock initialized: now 1786173341181808 us; error 48 us; skew 500 ppm
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:41.184326 15095 init.cc:377] Logged in from keytab as kudu/127.22.2.189@KRBTEST.COM (short username kudu)
I20260808 07:15:41.184775 15095 webserver.cc:460] Webserver started at http://127.22.2.189:42715/ using document root <none> and password file <none>
I20260808 07:15:41.184993 15095 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:41.185058 15095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:41.185200 15095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:41.186237 15095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/data/instance:
uuid: "25ecd36189d74314a4ad3e3d0fbd76d4"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "67ef47b0fc226da28d7e392f15a475a5"
server_key_iv: "9127cb6070c90d781b98708a4c9bd779"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.186625 15095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/wal/instance:
uuid: "25ecd36189d74314a4ad3e3d0fbd76d4"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "67ef47b0fc226da28d7e392f15a475a5"
server_key_iv: "9127cb6070c90d781b98708a4c9bd779"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.187978 15095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:41.188709 15111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:41.188882 15095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:41.188983 15095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/wal
uuid: "25ecd36189d74314a4ad3e3d0fbd76d4"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "67ef47b0fc226da28d7e392f15a475a5"
server_key_iv: "9127cb6070c90d781b98708a4c9bd779"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.189054 15095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:41.194409 15087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } attempt: 2
I20260808 07:15:41.194808 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.194684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43166)
0808 07:15:41.194748 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.194782 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:15:41.195139 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.195042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43168)
0808 07:15:41.195088 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.195115 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:15:41.195225 15087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:37203: Network error: Client connection negotiation failed: client connection to 127.22.2.189:37203: connect: Connection refused (error 111)
I20260808 07:15:41.203279 15095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:41.205562 15095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:41.205718 15095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:41.210060 15095 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:37203
I20260808 07:15:41.210099 15166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:37203 every 8 connection(s)
I20260808 07:15:41.210453 15095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
I20260808 07:15:41.211424 15167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:41.212957 15167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:41.215071 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 15095
I20260808 07:15:41.215225 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-1/wal/instance
I20260808 07:15:41.215418 22538 external_mini_cluster.cc:1428] Setting key 4dc56d9ad6084788a75413053f8e5f8f
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:41.222994 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.213583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.22.2.190:44351)
0808 07:15:41.213669 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.213671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.213857 (+ 186us) server_negotiation.cc:407] Connection header received
0808 07:15:41.214382 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.214384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.214400 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.214462 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.215161 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.215829 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.217283 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.217635 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.217712 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.219902 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.219904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.219905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.219914 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.221628 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.222017 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.222018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.222019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.222089 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.222327 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.222328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.222329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.222488 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.222564 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.222799 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.222890 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:15:41.223021 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.213570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44351 (local address 127.0.0.1:60344)
0808 07:15:41.213742 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.213776 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.214253 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.214533 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.214536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.214768 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.215040 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.215046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.215924 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.215926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.217190 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.217194 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.217742 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.217743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.217855 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.218363 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.218371 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.219806 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.221723 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.221729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.221731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.221943 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.222167 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.222167 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.222168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.222260 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.222558 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.222562 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.222667 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.222747 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.222864 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
I20260808 07:15:41.223635 15167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.2.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.188@KRBTEST.COM" created.
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 07:15:41.231119 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.223969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43174 (local address 127.22.2.189:37203)
0808 07:15:41.224123 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.224127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.224146 (+ 19us) server_negotiation.cc:407] Connection header received
0808 07:15:41.224173 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.224175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.224186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.224246 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.224571 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.225238 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.225965 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.226257 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.226315 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.228379 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.228388 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.228388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.228396 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.229978 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.230393 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.230396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.230396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.230446 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.230645 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.230645 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.230645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.230767 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.230837 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.230982 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.231048 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260808 07:15:41.231292 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.223916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43174)
0808 07:15:41.223976 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.223989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.224046 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.224268 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.224269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.224418 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.224478 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.224480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.225352 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.225353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.225897 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.225899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.226323 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.226323 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.226409 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.226769 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.226773 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.228291 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.230065 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.230066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.230066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.230301 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.230509 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.230510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.230511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.230583 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.230832 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.230833 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.230881 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.230961 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.231139 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":69}
I20260808 07:15:41.231950 15167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:41.232293 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.232205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34270)
0808 07:15:41.232238 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.232277 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 07:15:41.232450 15115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
I20260808 07:15:41.232882 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.232795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34274)
0808 07:15:41.232844 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.232867 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:15:41.233004 15167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:36163: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
I20260808 07:15:41.242971 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:36163
--webserver_interface=127.22.2.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:35733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:44351,127.22.2.189:37203,127.22.2.188:36163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:15:41.282264 15167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } attempt: 1
I20260808 07:15:41.282832 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.282674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34286)
0808 07:15:41.282742 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.282788 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:15:41.283227 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.283121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34288)
0808 07:15:41.283173 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.283202 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 07:15:41.283356 15167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:36163: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
I20260808 07:15:41.286367 15087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } attempt: 3
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 07:15:41.293746 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.286664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43188)
0808 07:15:41.286711 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.286721 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.286773 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.286942 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.286943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.287092 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.287162 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.287163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.288121 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.288122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.288646 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.288647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.289123 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.289124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.289189 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.289646 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.289651 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.291054 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.292841 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.292842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.292842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.293007 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.293204 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.293205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.293205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.293257 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.293511 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.293512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.293550 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.293596 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.293674 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 07:15:41.293789 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.286755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.22.2.189:37203)
0808 07:15:41.286796 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.286798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.286809 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:15:41.286843 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.286844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.286855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.286918 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.287234 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.288044 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.288725 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.289050 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.289120 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.291129 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.291131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.291131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.291139 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.292760 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.293077 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.293078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.293079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.293142 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.293314 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.293315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.293315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.293451 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.293517 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.293627 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.293702 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:15:41.294273 15087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:41.294596 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.294511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34292)
0808 07:15:41.294545 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.294575 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 07:15:41.294971 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.294874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34296)
0808 07:15:41.294921 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.294946 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:15:41.295086 15087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:36163: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
I20260808 07:15:41.326278 15087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } attempt: 1
I20260808 07:15:41.326783 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.326608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34300)
0808 07:15:41.326685 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.326740 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:15:41.327200 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.327097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34302)
0808 07:15:41.327147 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.327178 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:15:41.327349 15087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:36163: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
W20260808 07:15:41.328088 15174 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:41.328277 15174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:41.328341 15174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:41.328382 15174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:41.329493 15174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:41.329531 15174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:41.329546 15174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:41.329597 15174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:41.329624 15174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:41.331089 15174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:35733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:44351,127.22.2.189:37203,127.22.2.188:36163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:36163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.2.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:41.331339 15174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:41.331624 15174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:41.336426 15183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:41.336455 15181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:41.336517 15180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:41.336735 15174 server_base.cc:1034] running on GCE node
I20260808 07:15:41.336977 15174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:41.337260 15174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:41.338428 15174 hybrid_clock.cc:648] HybridClock initialized: now 1786173341338401 us; error 52 us; skew 500 ppm
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:41.340917 15174 init.cc:377] Logged in from keytab as kudu/127.22.2.188@KRBTEST.COM (short username kudu)
I20260808 07:15:41.341331 15174 webserver.cc:460] Webserver started at http://127.22.2.188:34241/ using document root <none> and password file <none>
I20260808 07:15:41.341555 15174 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:41.341625 15174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:41.341779 15174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:41.342813 15174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/data/instance:
uuid: "84d732db2c524b9b9481d00272e3de45"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "ae83d3ded704afa6cf693371c868dd25"
server_key_iv: "740bcf4165962db95f7fd2cf264505ad"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.343185 15174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/wal/instance:
uuid: "84d732db2c524b9b9481d00272e3de45"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "ae83d3ded704afa6cf693371c868dd25"
server_key_iv: "740bcf4165962db95f7fd2cf264505ad"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.344640 15174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:41.345376 15191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:41.345546 15174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:41.345647 15174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/wal
uuid: "84d732db2c524b9b9481d00272e3de45"
format_stamp: "Formatted at 2026-08-08 07:15:41 on dist-test-slave-6k22"
server_key: "ae83d3ded704afa6cf693371c868dd25"
server_key_iv: "740bcf4165962db95f7fd2cf264505ad"
server_key_version: "encryptionkey@0"
I20260808 07:15:41.345731 15174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:41.351516 15167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } attempt: 2
I20260808 07:15:41.351943 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.351815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34316)
0808 07:15:41.351876 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.351913 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:15:41.352253 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.352175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34328)
0808 07:15:41.352212 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.352235 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 07:15:41.352375 15167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:36163: Network error: Client connection negotiation failed: client connection to 127.22.2.188:36163: connect: Connection refused (error 111)
I20260808 07:15:41.358664 15174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:41.361008 15174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:41.361116 15174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:41.365176 15174 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:36163
I20260808 07:15:41.365206 15250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:36163 every 8 connection(s)
I20260808 07:15:41.365553 15174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
I20260808 07:15:41.366511 15251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:41.368124 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 15174
I20260808 07:15:41.368249 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173068742558-22538-0/minicluster-data/master-2/wal/instance
I20260808 07:15:41.368093 15251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:41.368433 22538 external_mini_cluster.cc:1428] Setting key 84a9f9f4fd2e858ce543195be242f70f
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 07:15:41.377748 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.368843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60356 (local address 127.22.2.190:44351)
0808 07:15:41.368900 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.368902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.369743 (+ 841us) server_negotiation.cc:407] Connection header received
0808 07:15:41.370008 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.370009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.370021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.370105 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.370701 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.371181 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.372133 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.372341 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.372428 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.374824 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.374825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.374826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.374833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.376186 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.376776 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.376777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.376777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.376834 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.377091 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.377092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.377092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.377194 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.377245 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.377573 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.377656 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:15:41.377902 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.368818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:44351 (local address 127.0.0.1:60356)
0808 07:15:41.369040 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.369089 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.369908 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.370130 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.370133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.370451 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.370607 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.370612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.371263 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.371264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.372046 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.372049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.372459 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.372461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.372574 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.373095 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.373107 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.374725 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.376280 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.376288 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.376290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.376689 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.376904 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.376905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.376905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.377022 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.377259 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.377264 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.377427 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.377521 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.377626 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":139,"threads_started":1}
I20260808 07:15:41.378466 15251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:41.379051 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.368973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.22.2.188:36163)
0808 07:15:41.369117 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.369122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.369218 (+ 96us) server_negotiation.cc:407] Connection header received
0808 07:15:41.369902 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.369912 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.369943 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.370055 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.370810 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.371649 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.372432 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.372772 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.372844 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.375370 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.375377 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.375378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.375384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.377324 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.377705 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.377708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.377708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.377768 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.378014 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.378015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.378015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.378175 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.378325 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.378649 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.378728 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:41.385290 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.378733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:37203 (local address 127.0.0.1:43204)
0808 07:15:41.378773 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.378782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.378814 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.379012 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.379012 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.379160 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.379199 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.379201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.380011 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.380012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.380485 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.380487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.380876 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.380876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.380940 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.381293 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.381296 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.382443 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.384212 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.384213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.384214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.384446 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.384671 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.384673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.384673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.384734 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.384992 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.384993 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.385033 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.385105 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.385197 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 07:15:41.385290 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.378823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.22.2.189:37203)
0808 07:15:41.378888 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.378890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.378898 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:41.378924 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.378925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.378940 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.379008 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.379301 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.379936 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.380568 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.380784 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.380875 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.382528 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.382530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.382530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.382537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.384134 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.384519 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.384520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.384520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.384569 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.384809 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.384810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.384810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.384926 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.385019 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.385131 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.385200 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:15:41.385684 15251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:41.387689 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.379807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.22.2.190:44351)
0808 07:15:41.379853 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.379855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.379892 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:15:41.379965 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.379966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.379974 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.380045 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.380519 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.380953 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.381730 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.381922 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.382006 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.384595 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.384597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.384597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.384604 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.386482 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.386812 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.386813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.386813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.386875 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.387076 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.387077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.387077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.387206 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.387369 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.387406 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.387477 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 07:15:41.392567 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.385944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34354)
0808 07:15:41.386001 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.386012 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.386068 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.386175 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.386176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.386324 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.386369 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.386371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.386929 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.386931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.387689 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.387692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.388117 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.388118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.388168 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.388510 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.388516 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.389753 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.391309 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.391310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.391311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.391513 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.391778 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.391779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.391779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.391855 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.392127 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.392127 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.392185 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.392261 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.392344 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:15:41.392719 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.385979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.22.2.188:36163)
0808 07:15:41.386010 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.386012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.386052 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:15:41.386124 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.386124 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.386131 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.386170 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.386437 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.386861 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.387782 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.388040 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.388093 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.389851 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.389852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.389852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.389859 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.391207 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.391640 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.391641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.391641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.391722 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.391923 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.391924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.391924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.392054 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.392132 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.392303 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.392423 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 07:15:41.394356 15251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45: Bootstrap starting.
I20260808 07:15:41.395460 15251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:41.395831 15251 log.cc:824] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:41.396633 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.388956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.22.2.189:37203)
0808 07:15:41.388990 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.388991 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.389023 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:15:41.389043 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.389043 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.389052 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.389095 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.389561 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.389994 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.390984 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.391229 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.391304 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.393925 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.393926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.393927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.393934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.395590 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.395919 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.395920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.395920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.395977 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.396145 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.396146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.396147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.396284 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.396432 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.396517 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.396574 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:15:41.396771 15251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45: No bootstrap required, opened a new log
I20260808 07:15:41.398168 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:41.398756 15251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } }
I20260808 07:15:41.398939 15251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d732db2c524b9b9481d00272e3de45, State: Initialized, Role: FOLLOWER
I20260808 07:15:41.399041 15251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } }
I20260808 07:15:41.399326 15257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.399398 15251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:41.399403 15257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:41.399084 15059 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ea76f38873644680bd05f33ef35b02eb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60372
W20260808 07:15:41.401279 15271 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:41.401340 15271 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:41.402452 15087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } attempt: 2
I20260808 07:15:41.401644 15174 master_runner.cc:186] Error getting master registration for 127.22.2.190:44351: OK, instance_id { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" instance_seqno: 1786173341059897 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:15:41.404559 15174 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 07:15:41.410323 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.402707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34364)
0808 07:15:41.402761 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.402771 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.402810 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.402947 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.402948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.403128 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.403175 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.403178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.404077 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.404078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.404903 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.404906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.405374 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.405375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.405443 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.405937 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.405943 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.407459 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.409215 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.409217 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.409217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.409398 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.409616 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.409617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.409617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.409676 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.409970 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.409970 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.410017 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.410115 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.410220 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:15:41.410326 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.402773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.22.2.188:36163)
0808 07:15:41.402824 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.402826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.402837 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:15:41.402861 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.402862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.402873 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.402939 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.403249 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.403986 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.404991 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.405284 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.405356 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.407572 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.407574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.407575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.407583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.409111 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.409484 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.409486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.409486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.409541 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.409748 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.409748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.409749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.409897 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.409978 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.410154 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.410225 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:15:41.411760 15087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb: Bootstrap starting.
I20260808 07:15:41.412411 15087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:41.412699 15087 log.cc:824] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:41.413446 15087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb: No bootstrap required, opened a new log
I20260808 07:15:41.414450 15087 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } }
I20260808 07:15:41.414563 15087 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea76f38873644680bd05f33ef35b02eb, State: Initialized, Role: FOLLOWER
I20260808 07:15:41.414681 15087 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } }
I20260808 07:15:41.414978 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.415035 15087 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:41.415055 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:41.415174 15059 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:15:41.415251 15059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:41.416162 15059 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } }
I20260808 07:15:41.416857 15059 leader_election.cc:288] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [CANDIDATE]: Term 1 election: Requested vote from peers 25ecd36189d74314a4ad3e3d0fbd76d4 (127.22.2.189:37203), 84d732db2c524b9b9481d00272e3de45 (127.22.2.188:36163)
W20260808 07:15:41.417133 15286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:41.417217 15286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:41.417495 15221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea76f38873644680bd05f33ef35b02eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "84d732db2c524b9b9481d00272e3de45"
I20260808 07:15:41.417623 15221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:41.418682 15221 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea76f38873644680bd05f33ef35b02eb in term 1.
I20260808 07:15:41.418924 15030 leader_election.cc:302] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d732db2c524b9b9481d00272e3de45, ea76f38873644680bd05f33ef35b02eb; no voters:
I20260808 07:15:41.419032 15274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:41.419154 15274 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [term 1 LEADER]: Becoming Leader. State: Replica: ea76f38873644680bd05f33ef35b02eb, State: Running, Role: LEADER
I20260808 07:15:41.419256 15274 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } }
I20260808 07:15:41.419695 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea76f38873644680bd05f33ef35b02eb. Latest consensus state: current_term: 1 leader_uuid: "ea76f38873644680bd05f33ef35b02eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.419783 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:41.418810 15010 master_runner.cc:186] Error getting master registration for 127.22.2.189:37203: OK, instance_id { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" instance_seqno: 1786173341208813 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:15:41.420917 15010 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:15:41.424117 15297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:41.424337 15297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:41.429520 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.418091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.22.2.190:44351)
0808 07:15:41.418144 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.418146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.421435 (+ 3289us) server_negotiation.cc:407] Connection header received
0808 07:15:41.421550 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.421551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.421563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.421595 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.422452 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.423211 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.424228 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.424528 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.424566 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.426266 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.426268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.426268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.426275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.428099 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.428539 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.428540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.428540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.428600 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.428823 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.428824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.428824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.428969 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.429063 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.429298 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.429390 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:15:41.433180 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.418384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.22.2.188:36163)
0808 07:15:41.418423 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.418426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.421435 (+ 3009us) server_negotiation.cc:407] Connection header received
0808 07:15:41.421852 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.421853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.421864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.421895 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.423201 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.423929 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.426716 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.427041 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.427089 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.429999 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.430001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.430001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.430010 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.431867 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.432270 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.432271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.432272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.432341 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.432559 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.432560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.432560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.432692 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.432763 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.432984 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.433062 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:15:41.433951 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.421213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.22.2.189:37203)
0808 07:15:41.421257 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.421259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.421531 (+ 272us) server_negotiation.cc:407] Connection header received
0808 07:15:41.421726 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.421727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.421739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.421769 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.424025 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.424686 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.427544 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.428396 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.428443 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.430926 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.430928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.430928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.430937 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.432653 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.433044 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.433045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.433046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.433101 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.433305 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.433306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.433306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.433436 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.433502 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.433744 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.433837 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:15:41.436614 15221 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [term 1 FOLLOWER]: Refusing update from remote peer ea76f38873644680bd05f33ef35b02eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:41.436977 15295 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:41.438388 15257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea76f38873644680bd05f33ef35b02eb. Latest consensus state: current_term: 1 leader_uuid: "ea76f38873644680bd05f33ef35b02eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.438488 15257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:41.439465 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea76f38873644680bd05f33ef35b02eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.439594 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:41.439697 15257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea76f38873644680bd05f33ef35b02eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.439778 15257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:41.440402 15301 mvcc.cc:205] Tried to move back new op lower bound from 7316166006521552896 to 7316166006454116352. Current Snapshot: MvccSnapshot[applied={T|T < 7316166006521552896}]
I20260808 07:15:41.443526 15167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } attempt: 3
I20260808 07:15:41.445627 15297 catalog_manager.cc:1347] Generated new cluster ID: a368539195f0434eaed3ff4ce7c96503
I20260808 07:15:41.445683 15297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:41 dist-test-slave-6k22 krb5kdc[14995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 07:15:41.451962 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.443930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34378 (local address 127.22.2.188:36163)
0808 07:15:41.443986 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.443989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.443997 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:41.444015 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.444016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.444024 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.444072 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.444377 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.444843 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.446263 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.446467 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.446546 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.448739 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:41.448740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:41.448741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:41.448749 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:41.450451 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.451053 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.451054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.451054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.451134 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:41.451387 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:41.451388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:41.451388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:41.451538 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:41.451614 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:41.451793 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 07:15:41.451866 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:15:41.451962 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:41.443827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:36163 (local address 127.0.0.1:34378)
0808 07:15:41.443887 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:41.443901 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:41.443945 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:41.444074 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:41.444074 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:41.444251 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:41.444295 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.444297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.445384 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.445385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.446193 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:41.446195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:41.446543 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:41.446543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:41.446600 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:41.447079 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:41.447085 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:41.448646 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:41.450633 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.450635 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.450635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.450882 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.451224 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:41.451225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:41.451225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:41.451309 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:41.451641 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:41.451642 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:41.451687 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:15:41.451755 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:15:41.451866 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:15:41.453760 15167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4: Bootstrap starting.
I20260808 07:15:41.454492 15167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:41.454811 15167 log.cc:824] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:41.455750 15167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4: No bootstrap required, opened a new log
I20260808 07:15:41.458141 15167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } }
I20260808 07:15:41.458277 15167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ecd36189d74314a4ad3e3d0fbd76d4, State: Initialized, Role: FOLLOWER
I20260808 07:15:41.458386 15167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } }
I20260808 07:15:41.458714 15167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:41.458787 15138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:41.459793 15138 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [term 1 FOLLOWER]: Refusing update from remote peer ea76f38873644680bd05f33ef35b02eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:41.459997 15139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea76f38873644680bd05f33ef35b02eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4"
I20260808 07:15:41.460107 15295 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:41.461143 15297 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:41.461218 15139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea76f38873644680bd05f33ef35b02eb in term 1.
I20260808 07:15:41.458684 15306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.461582 15306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:41.461974 15297 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:41.462330 15306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea76f38873644680bd05f33ef35b02eb. Latest consensus state: current_term: 1 leader_uuid: "ea76f38873644680bd05f33ef35b02eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.462453 15306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:41.462508 15317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea76f38873644680bd05f33ef35b02eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.462827 15317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:41.462953 15318 mvcc.cc:205] Tried to move back new op lower bound from 7316166006521552896 to 7316166006454116352. Current Snapshot: MvccSnapshot[applied={T|T < 7316166006521552896}]
W20260808 07:15:41.464682 15321 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:41.464901 15321 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:41.465224 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea76f38873644680bd05f33ef35b02eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea76f38873644680bd05f33ef35b02eb" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 44351 } } peers { permanent_uuid: "25ecd36189d74314a4ad3e3d0fbd76d4" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 37203 } } peers { permanent_uuid: "84d732db2c524b9b9481d00272e3de45" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 36163 } } }
I20260808 07:15:41.465329 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:41.483649 15297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea76f38873644680bd05f33ef35b02eb: Generated new TSK 0
I20260808 07:15:41.483937 15297 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:15:41.514999 15090 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.513378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.22.2.190:44351)
0808 07:15:41.513426 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.513428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.514287 (+ 859us) server_negotiation.cc:407] Connection header received
0808 07:15:41.514397 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.514398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.514410 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.514441 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.514921 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60386: BlockingRecv error: recv got EOF from 127.0.0.1:60386 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:15:41.515892 15255 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.513722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.22.2.188:36163)
0808 07:15:41.513764 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.513765 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.515318 (+ 1553us) server_negotiation.cc:407] Connection header received
0808 07:15:41.515421 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.515422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.515432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.515461 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.515803 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34388: BlockingRecv error: recv got EOF from 127.0.0.1:34388 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:15:41.516677 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.514067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.22.2.189:37203)
0808 07:15:41.514113 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.514115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.516165 (+ 2050us) server_negotiation.cc:407] Connection header received
0808 07:15:41.516261 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.516262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.516274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.516306 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.516613 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43234: BlockingRecv error: recv got EOF from 127.0.0.1:43234 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 07:15:41.519137 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.518327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43244 (local address 127.22.2.189:37203)
0808 07:15:41.518383 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.518384 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.518623 (+ 239us) server_negotiation.cc:407] Connection header received
0808 07:15:41.518728 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.518729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.518746 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.518775 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.519091 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43244: BlockingRecv error: recv got EOF from 127.0.0.1:43244 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:41.522106 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.521373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.22.2.189:37203)
0808 07:15:41.521406 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.521407 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.521632 (+ 225us) server_negotiation.cc:407] Connection header received
0808 07:15:41.521720 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.521721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.521730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.521758 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.522062 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43248: BlockingRecv error: recv got EOF from 127.0.0.1:43248 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:41.525959 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.525198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43258 (local address 127.22.2.189:37203)
0808 07:15:41.525234 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.525236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.525446 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:15:41.525535 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.525536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.525545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.525573 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.525918 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43258: BlockingRecv error: recv got EOF from 127.0.0.1:43258 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:15:41.531900 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.531102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.22.2.189:37203)
0808 07:15:41.531135 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.531137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.531374 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:15:41.531466 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.531467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.531477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.531504 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.531855 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43266: BlockingRecv error: recv got EOF from 127.0.0.1:43266 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:41.541787 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.540985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.22.2.189:37203)
0808 07:15:41.541022 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.541023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.541259 (+ 236us) server_negotiation.cc:407] Connection header received
0808 07:15:41.541355 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.541356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.541366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.541394 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.541740 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43272: BlockingRecv error: recv got EOF from 127.0.0.1:43272 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:41.560379 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.559254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.22.2.189:37203)
0808 07:15:41.559297 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.559299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.559633 (+ 334us) server_negotiation.cc:407] Connection header received
0808 07:15:41.559771 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.559772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.559784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.559824 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.560319 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43274: BlockingRecv error: recv got EOF from 127.0.0.1:43274 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:15:41.594812 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.593780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.22.2.189:37203)
0808 07:15:41.593838 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.593841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.594165 (+ 324us) server_negotiation.cc:407] Connection header received
0808 07:15:41.594278 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.594279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.594292 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.594338 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.594750 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43284: BlockingRecv error: recv got EOF from 127.0.0.1:43284 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:15:41.661032 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.660308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.22.2.189:37203)
0808 07:15:41.660361 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.660364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.660492 (+ 128us) server_negotiation.cc:407] Connection header received
0808 07:15:41.660535 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.660536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.660547 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.660601 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.660969 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43288: BlockingRecv error: recv got EOF from 127.0.0.1:43288 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:15:41.791154 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:41.790344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.22.2.189:37203)
0808 07:15:41.790433 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:41.790435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:41.790536 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:15:41.790636 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:41.790638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:41.790650 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:41.790730 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:41.791084 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43296: BlockingRecv error: recv got EOF from 127.0.0.1:43296 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 07:15:42.049094 15171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:42.048446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43304 (local address 127.22.2.189:37203)
0808 07:15:42.048497 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:42.048499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:42.048525 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:15:42.048600 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:42.048601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:42.048614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:42.048664 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:42.049026 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43304: BlockingRecv error: recv got EOF from 127.0.0.1:43304 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:15:42.401798 15271 catalog_manager.cc:1259] Loaded cluster ID: a368539195f0434eaed3ff4ce7c96503
I20260808 07:15:42.401881 15271 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45: loading cluster ID for follower catalog manager: success
I20260808 07:15:42.402647 15271 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45: acquiring CA information for follower catalog manager: success
I20260808 07:15:42.402901 15271 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84d732db2c524b9b9481d00272e3de45: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:15:42.465404 15321 catalog_manager.cc:1259] Loaded cluster ID: a368539195f0434eaed3ff4ce7c96503
I20260808 07:15:42.465485 15321 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4: loading cluster ID for follower catalog manager: success
I20260808 07:15:42.466240 15321 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4: acquiring CA information for follower catalog manager: success
I20260808 07:15:42.466495 15321 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25ecd36189d74314a4ad3e3d0fbd76d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:15:42.565129 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:42.562200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.22.2.189:37203)
0808 07:15:42.562331 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:42.562333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:42.562412 (+ 79us) server_negotiation.cc:407] Connection header received
0808 07:15:42.562518 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:42.562519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:42.562607 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:42.562700 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:42.562880 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.563383 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.564279 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.564492 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.564578 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:42.564755 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:42.564878 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:42.564961 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:42.565001 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:15:42.565024 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":55,"threads_started":1}
I20260808 07:15:42.569548 15403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:42.566981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34396 (local address 127.22.2.188:36163)
0808 07:15:42.567104 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:42.567107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:42.567116 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:15:42.567195 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:42.567196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:42.567244 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:42.567324 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:42.567457 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.567997 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.568728 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.568953 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.569038 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:42.569188 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:42.569289 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:42.569365 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:42.569433 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:15:42.569457 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260808 07:15:42.573770 15403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:42.571419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.22.2.188:36163)
0808 07:15:42.571453 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:42.571455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:42.571616 (+ 161us) server_negotiation.cc:407] Connection header received
0808 07:15:42.571679 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:42.571679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:42.571690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:42.571736 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:42.571873 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.572432 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.573150 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.573341 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.573422 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:42.573554 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:42.573583 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:42.573631 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:42.573686 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 07:15:42.573705 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:42.574294 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:42.571185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.22.2.189:37203)
0808 07:15:42.571228 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:42.571230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:42.571351 (+ 121us) server_negotiation.cc:407] Connection header received
0808 07:15:42.571473 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:42.571474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:42.571486 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:42.571516 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:42.571754 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.572467 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.573508 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.573797 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.573862 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:42.574009 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:42.574039 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:42.574066 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:42.574180 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 07:15:42.574200 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:15:42.574810 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:42.571246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.22.2.190:44351)
0808 07:15:42.571879 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:42.571881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:42.571889 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:42.571927 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:42.571928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:42.571988 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:42.572071 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:42.572200 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.572933 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.573934 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:42.574242 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:42.574311 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:42.574499 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:42.574576 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:42.574643 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:42.574707 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:15:42.574726 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":62,"threads_started":1}
I20260808 07:15:42.576445 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 15010
I20260808 07:15:42.585728 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 15095
I20260808 07:15:42.593025 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 15174
2026-08-08T07:15:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2902 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:42 dist-test-slave-6k22 krb5kdc[15415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:42 dist-test-slave-6k22 krb5kdc[15415](info): set up 2 sockets
Aug 08 07:15:42 dist-test-slave-6k22 krb5kdc[15415](info): commencing operation
krb5kdc: starting...
W20260808 07:15:43.727056 22538 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/test-user.keytab.
Aug 08 07:15:43 dist-test-slave-6k22 krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.2.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
Entry for principal kudu/127.22.2.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab.
I20260808 07:15:43.852401 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42049
--webserver_interface=127.22.2.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42049,127.22.2.189:33847,127.22.2.188:46299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:43.934957 15430 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:43.935165 15430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:43.935184 15430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:43.935236 15430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:43.936462 15430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:43.936507 15430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:43.936522 15430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:43.936569 15430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:43.936589 15430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:43.938683 15430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42049,127.22.2.189:33847,127.22.2.188:46299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.190.keytab
--principal=kudu/127.22.2.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.190:42049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.2.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:43.938994 15430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:43.939292 15430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:43.944820 15438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:43.944828 15435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:43.944828 15436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:43.945199 15430 server_base.cc:1034] running on GCE node
I20260808 07:15:43.945472 15430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:43.945783 15430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:43.946951 15430 hybrid_clock.cc:648] HybridClock initialized: now 1786173343946947 us; error 37 us; skew 500 ppm
Aug 08 07:15:43 dist-test-slave-6k22 krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173343, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:43.954679 15430 init.cc:377] Logged in from keytab as kudu/127.22.2.190@KRBTEST.COM (short username kudu)
I20260808 07:15:43.955081 15430 webserver.cc:460] Webserver started at http://127.22.2.190:43315/ using document root <none> and password file <none>
I20260808 07:15:43.955283 15430 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:43.955331 15430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:43.955425 15430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:43.956508 15430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/data/instance:
uuid: "de2ec9df307f4869872afb37e316568a"
format_stamp: "Formatted at 2026-08-08 07:15:43 on dist-test-slave-6k22"
server_key: "a7a75b27090c03d5d8d097baafd1b324"
server_key_iv: "9b3ab4ac16d916dd84daf28e7355c404"
server_key_version: "encryptionkey@0"
I20260808 07:15:43.956806 15430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/wal/instance:
uuid: "de2ec9df307f4869872afb37e316568a"
format_stamp: "Formatted at 2026-08-08 07:15:43 on dist-test-slave-6k22"
server_key: "a7a75b27090c03d5d8d097baafd1b324"
server_key_iv: "9b3ab4ac16d916dd84daf28e7355c404"
server_key_version: "encryptionkey@0"
I20260808 07:15:43.958065 15430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:43.958775 15446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:43.958936 15430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:43.959030 15430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/wal
uuid: "de2ec9df307f4869872afb37e316568a"
format_stamp: "Formatted at 2026-08-08 07:15:43 on dist-test-slave-6k22"
server_key: "a7a75b27090c03d5d8d097baafd1b324"
server_key_iv: "9b3ab4ac16d916dd84daf28e7355c404"
server_key_version: "encryptionkey@0"
I20260808 07:15:43.959110 15430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:43.991545 15430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:43.993526 15430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:43.993703 15430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:43.998140 15430 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.190:42049
I20260808 07:15:43.998179 15500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.190:42049 every 8 connection(s)
I20260808 07:15:43.998589 15430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:43.999593 15501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:44.001178 15501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173343, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:44.008018 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 15430
I20260808 07:15:44.008133 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-0/wal/instance
I20260808 07:15:44.008390 22538 external_mini_cluster.cc:1428] Setting key 8d8d710d232629fff2fabd9085fb990e
I20260808 07:15:44.011137 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.001900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.22.2.190:42049)
0808 07:15:44.002091 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.002095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.002129 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:15:44.002630 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.002632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.002649 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.002727 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.003239 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.004151 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.004963 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.005257 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.005331 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.007376 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.007383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.007383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.007389 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.009439 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.009805 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.009808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.009808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.009929 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.010138 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.010139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.010139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.010443 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.010547 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.010726 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.010870 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":108,"threads_started":1}
I20260808 07:15:44.011153 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.001845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42049 (local address 127.0.0.1:51958)
0808 07:15:44.002020 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.002068 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.002497 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.002759 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.002761 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.003004 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.003154 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.003158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.004230 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.004232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.004890 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.004892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.005359 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.005360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.005414 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.005929 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.005937 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.007279 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.009531 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.009538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.009539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.009728 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.009996 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.009997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.009997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.010075 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.010529 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.010534 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.010603 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.010701 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.010934 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":100,"thread_start_us":76,"threads_started":1}
I20260808 07:15:44.012094 15501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:44.012429 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.012310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53766)
0808 07:15:44.012366 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.012407 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 07:15:44.012612 15449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
I20260808 07:15:44.013072 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.012996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53770)
0808 07:15:44.013031 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.013055 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 07:15:44.013191 15501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:33847: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.2.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
Entry for principal kudu/127.22.2.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab.
I20260808 07:15:44.034633 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:33847
--webserver_interface=127.22.2.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42049,127.22.2.189:33847,127.22.2.188:46299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:15:44.062398 15501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } attempt: 1
I20260808 07:15:44.062860 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.062711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53784)
0808 07:15:44.062775 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.062819 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:15:44.063256 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.063151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53796)
0808 07:15:44.063196 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.063224 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:15:44.063345 15501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:33847: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
W20260808 07:15:44.126609 15508 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:44.126850 15508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:44.126874 15508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:44.126941 15508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:44.128634 15508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:44.128686 15508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:44.128705 15508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:44.128760 15508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:44.128795 15508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:44.130503 15508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42049,127.22.2.189:33847,127.22.2.188:46299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.189.keytab
--principal=kudu/127.22.2.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.189:33847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.2.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:44.130792 15508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:44.131057 15508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:15:44.131486 15501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } attempt: 2
I20260808 07:15:44.131932 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.131795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53812)
0808 07:15:44.131863 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.131906 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:15:44.132432 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.132305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53816)
0808 07:15:44.132372 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.132407 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 07:15:44.132540 15501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.189:33847: Network error: Client connection negotiation failed: client connection to 127.22.2.189:33847: connect: Connection refused (error 111)
W20260808 07:15:44.136667 15517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:44.136677 15514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:44.136847 15513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:44.136947 15508 server_base.cc:1034] running on GCE node
I20260808 07:15:44.137225 15508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:44.137565 15508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:44.138768 15508 hybrid_clock.cc:648] HybridClock initialized: now 1786173344138744 us; error 31 us; skew 500 ppm
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173344, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:44.141511 15508 init.cc:377] Logged in from keytab as kudu/127.22.2.189@KRBTEST.COM (short username kudu)
I20260808 07:15:44.142057 15508 webserver.cc:460] Webserver started at http://127.22.2.189:39571/ using document root <none> and password file <none>
I20260808 07:15:44.142376 15508 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:44.142454 15508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:44.142627 15508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:44.143842 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/data/instance:
uuid: "d155d4a0abea45a0bb4a17d8690ac238"
format_stamp: "Formatted at 2026-08-08 07:15:44 on dist-test-slave-6k22"
server_key: "d218f08d25604ca56a80235e70243e16"
server_key_iv: "2a70722b99a53ccef6633a8ffa3cc945"
server_key_version: "encryptionkey@0"
I20260808 07:15:44.144243 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/wal/instance:
uuid: "d155d4a0abea45a0bb4a17d8690ac238"
format_stamp: "Formatted at 2026-08-08 07:15:44 on dist-test-slave-6k22"
server_key: "d218f08d25604ca56a80235e70243e16"
server_key_iv: "2a70722b99a53ccef6633a8ffa3cc945"
server_key_version: "encryptionkey@0"
I20260808 07:15:44.145761 15508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:44.146565 15525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:44.146756 15508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:44.146871 15508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/wal
uuid: "d155d4a0abea45a0bb4a17d8690ac238"
format_stamp: "Formatted at 2026-08-08 07:15:44 on dist-test-slave-6k22"
server_key: "d218f08d25604ca56a80235e70243e16"
server_key_iv: "2a70722b99a53ccef6633a8ffa3cc945"
server_key_version: "encryptionkey@0"
I20260808 07:15:44.146950 15508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:44.180238 15508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:44.183421 15508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:44.183594 15508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:44.188887 15508 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.189:33847
I20260808 07:15:44.188923 15587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.189:33847 every 8 connection(s)
I20260808 07:15:44.189321 15508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/data/info.pb
I20260808 07:15:44.190330 15588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:44.192014 15588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:44.192863 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 15508
I20260808 07:15:44.192989 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-1/wal/instance
I20260808 07:15:44.193225 22538 external_mini_cluster.cc:1428] Setting key f832daa70f4a668f40aa09745a0e143c
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173344, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.2.188@KRBTEST.COM; defaulting to no policy
I20260808 07:15:44.201763 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.192754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51972 (local address 127.22.2.190:42049)
0808 07:15:44.192827 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.192830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.193067 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:15:44.193651 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.193653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.193666 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.193717 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.194367 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.195001 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.196104 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.196444 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.196520 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.198657 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.198659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.198659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.198667 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.200388 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.200773 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.200775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.200775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.200853 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.201091 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.201092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.201093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.201245 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.201323 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.201552 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.201638 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:15:44.201884 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.192729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42049 (local address 127.0.0.1:51972)
0808 07:15:44.192934 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.192976 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.193533 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.193788 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.193791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.194032 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.194235 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.194239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.195113 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.195114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.196016 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.196019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.196556 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.196557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.196665 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.197177 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.197187 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.198566 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.200473 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.200479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.200481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.200695 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.200928 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.200929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.200929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.201023 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.201317 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.201321 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.201429 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.201509 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.201623 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":126,"threads_started":1}
I20260808 07:15:44.202415 15588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.2.188@KRBTEST.COM" created.
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173344, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 07:15:44.209839 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.202683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53822)
0808 07:15:44.202730 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.202742 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.202786 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.203225 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.203225 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.203338 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.203382 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.203383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.204232 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.204233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.204773 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.204775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.205372 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.205372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.205447 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.205869 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.205874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.207003 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.208767 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.208768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.208768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.208938 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.209152 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.209152 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.209153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.209209 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.209514 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.209515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.209553 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.209619 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.209719 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:15:44.209867 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.202756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.22.2.189:33847)
0808 07:15:44.203090 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.203094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.203111 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:15:44.203130 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.203131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.203142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.203207 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.203448 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.204147 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.204857 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.205220 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.205288 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.207087 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.207095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.207096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.207102 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.208662 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.209021 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.209024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.209024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.209083 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.209283 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.209284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.209284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.209431 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.209520 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.209655 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.209772 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":306,"thread_start_us":121,"threads_started":1}
I20260808 07:15:44.210453 15588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:44.210777 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.210667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40440)
0808 07:15:44.210717 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.210757 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:15:44.210954 15526 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
I20260808 07:15:44.211385 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.211294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40448)
0808 07:15:44.211343 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.211368 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:15:44.211510 15588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:46299: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
Entry for principal kudu/127.22.2.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab.
I20260808 07:15:44.222057 22538 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyv8zyi/build/release/bin/kudu
/tmp/dist-test-taskyv8zyi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:46299
--webserver_interface=127.22.2.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.2.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.2.190:42049,127.22.2.189:33847,127.22.2.188:46299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:15:44.223683 15501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } attempt: 3
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173343, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 07:15:44.231159 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.224002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53830)
0808 07:15:44.224057 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.224067 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.224114 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.224264 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.224265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.224455 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.224529 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.224531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.225169 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.225171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.226078 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.226081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.226471 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.226472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.226562 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.227001 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.227008 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.228409 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.230085 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.230086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.230087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.230268 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.230483 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.230484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.230484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.230551 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.230812 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.230813 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.230863 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.230954 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.231060 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:15:44.231158 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.224063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.22.2.189:33847)
0808 07:15:44.224123 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.224125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.224135 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:44.224175 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.224176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.224194 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.224247 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.224610 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.225084 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.226174 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.226399 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.226449 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.228505 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.228506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.228507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.228515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.229985 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.230351 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.230352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.230353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.230415 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.230627 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.230627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.230628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.230743 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.230818 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.230979 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.231059 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:15:44.231597 15501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:44.231973 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.231873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40456)
0808 07:15:44.231922 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.231953 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:15:44.232306 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.232212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40466)
0808 07:15:44.232262 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.232285 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:15:44.232406 15501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:46299: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
I20260808 07:15:44.260736 15588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } attempt: 1
I20260808 07:15:44.261196 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.261068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40468)
0808 07:15:44.261129 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.261168 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:15:44.261565 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.261440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40470)
0808 07:15:44.261512 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.261537 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 07:15:44.261656 15588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:46299: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
I20260808 07:15:44.263532 15501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } attempt: 1
I20260808 07:15:44.263931 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.263807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40482)
0808 07:15:44.263874 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.263904 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 07:15:44.264317 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.264234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40492)
0808 07:15:44.264275 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.264300 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:15:44.264437 15501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:46299: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
W20260808 07:15:44.304896 15596 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:44.305111 15596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:44.305128 15596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:44.305147 15596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:44.306202 15596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:44.306237 15596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:44.306252 15596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:44.306264 15596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:44.306277 15596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:44.307705 15596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.2.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.2.190:42049,127.22.2.189:33847,127.22.2.188:46299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/krb5kdc/kudu_127.22.2.188.keytab
--principal=kudu/127.22.2.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.2.188:46299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.2.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:15:44.307899 15596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:44.308169 15596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:44.312929 15605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:44.312992 15603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:44.312981 15602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:44.313169 15596 server_base.cc:1034] running on GCE node
I20260808 07:15:44.313412 15596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:44.313742 15596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:44.314899 15596 hybrid_clock.cc:648] HybridClock initialized: now 1786173344314881 us; error 36 us; skew 500 ppm
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173344, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:44.317493 15596 init.cc:377] Logged in from keytab as kudu/127.22.2.188@KRBTEST.COM (short username kudu)
I20260808 07:15:44.317916 15596 webserver.cc:460] Webserver started at http://127.22.2.188:44009/ using document root <none> and password file <none>
I20260808 07:15:44.318146 15596 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:44.318213 15596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:44.318390 15596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:44.319433 15596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/data/instance:
uuid: "2e1c4adf66d3423095bb4ba7bdca9907"
format_stamp: "Formatted at 2026-08-08 07:15:44 on dist-test-slave-6k22"
server_key: "f3f64fd2b0734b70f3339c0dccfbbdfa"
server_key_iv: "f0557d1a99460e564dc0892edd6c50b0"
server_key_version: "encryptionkey@0"
I20260808 07:15:44.319866 15596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/wal/instance:
uuid: "2e1c4adf66d3423095bb4ba7bdca9907"
format_stamp: "Formatted at 2026-08-08 07:15:44 on dist-test-slave-6k22"
server_key: "f3f64fd2b0734b70f3339c0dccfbbdfa"
server_key_iv: "f0557d1a99460e564dc0892edd6c50b0"
server_key_version: "encryptionkey@0"
I20260808 07:15:44.321177 15596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:44.321946 15612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:44.322234 15596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:44.322345 15596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/data,/tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/wal
uuid: "2e1c4adf66d3423095bb4ba7bdca9907"
format_stamp: "Formatted at 2026-08-08 07:15:44 on dist-test-slave-6k22"
server_key: "f3f64fd2b0734b70f3339c0dccfbbdfa"
server_key_iv: "f0557d1a99460e564dc0892edd6c50b0"
server_key_version: "encryptionkey@0"
I20260808 07:15:44.322419 15596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:44.329787 15588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } attempt: 2
I20260808 07:15:44.330188 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.330067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40508)
0808 07:15:44.330125 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.330160 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:15:44.330551 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.330455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40514)
0808 07:15:44.330501 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.330525 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:15:44.330663 15588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.2.188:46299: Network error: Client connection negotiation failed: client connection to 127.22.2.188:46299: connect: Connection refused (error 111)
I20260808 07:15:44.332036 15596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:44.334230 15596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:44.334393 15596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:44.338476 15596 rpc_server.cc:307] RPC server started. Bound to: 127.22.2.188:46299
I20260808 07:15:44.338507 15668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.2.188:46299 every 8 connection(s)
I20260808 07:15:44.338840 15596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/data/info.pb
I20260808 07:15:44.339648 15501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } attempt: 2
I20260808 07:15:44.339962 15669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:44.342161 15669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173343, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.190@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173344, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:44.347611 22538 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyv8zyi/build/release/bin/kudu as pid 15596
I20260808 07:15:44.347756 22538 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyv8zyi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173068742558-22538-0/minicluster-data/master-2/wal/instance
I20260808 07:15:44.348004 22538 external_mini_cluster.cc:1428] Setting key d9dc65f89a59615ad919b627e6d197d0
I20260808 07:15:44.349344 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.339939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40526)
0808 07:15:44.340001 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.340013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.340063 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.341255 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.341257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.341459 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.341518 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.341521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.342642 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.342644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.343160 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.343162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.343827 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.343828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.343885 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.344342 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.344347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.345593 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.347620 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.347622 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.347623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.347865 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.348409 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.348409 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.348410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.348474 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.348942 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.348943 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.348984 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.349116 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.349239 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:15:44.349593 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.340119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40526 (local address 127.22.2.188:46299)
0808 07:15:44.340301 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.340310 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.340349 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:15:44.341029 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.341039 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.341102 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.341232 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.341627 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.342561 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.343266 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.343747 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.343869 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.345678 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.345684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.345684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.345694 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.347338 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.348233 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.348235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.348235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.348310 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.348550 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.348551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.348551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.348860 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.348951 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.349203 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.349350 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260808 07:15:44.350668 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.342836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.190:42049 (local address 127.0.0.1:51984)
0808 07:15:44.342978 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.343014 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.343102 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.343233 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.343236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.343494 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.343602 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.343604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.344209 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.344210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.344805 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.344807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.345259 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.345261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.345344 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.345930 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.345934 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.347309 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.349301 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.349308 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.349310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.349580 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.349816 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.349817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.349817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.349963 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.350313 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.350318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.350376 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.350471 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.350580 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260808 07:15:44.350687 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.342859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.22.2.190:42049)
0808 07:15:44.342904 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.342906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.343086 (+ 180us) server_negotiation.cc:407] Connection header received
0808 07:15:44.343150 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.343151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.343162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.343207 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.343683 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.344124 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.344908 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.345138 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.345222 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.347422 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.347424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.347424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.347431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.349212 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.349665 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.349666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.349667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.349748 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.350062 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.350064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.350064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.350207 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.350283 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.350520 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.350603 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:15:44.351126 15669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:44.351889 15501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a: Bootstrap starting.
I20260808 07:15:44.352597 15501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:44.352945 15501 log.cc:824] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:44.354214 15501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a: No bootstrap required, opened a new log
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173344, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
I20260808 07:15:44.356451 15501 raft_consensus.cc:348] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } }
I20260808 07:15:44.356597 15501 raft_consensus.cc:729] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2ec9df307f4869872afb37e316568a, State: Initialized, Role: FOLLOWER
I20260808 07:15:44.356684 15501 consensus_queue.cc:260] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } }
I20260808 07:15:44.357044 15501 sys_catalog.cc:564] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:44.357052 15677 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.357152 15677 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:15:44.358964 15689 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:44.359022 15689 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:44.359050 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.348558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40530 (local address 127.22.2.188:46299)
0808 07:15:44.349076 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.349078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.349085 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:44.349105 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.349105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.349117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.349184 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.350611 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.351662 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.352470 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.352707 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.352769 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.355096 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.355097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.355098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.355105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.357359 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.357832 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.357833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.357834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.357889 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.358163 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.358164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.358164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.358290 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.358457 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.358832 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.358918 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":74,"threads_started":1}
I20260808 07:15:44.359287 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.351473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.22.2.189:33847)
0808 07:15:44.351514 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.351516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.351524 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:44.351548 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.351549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.351577 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.351634 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.351946 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.352616 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.353502 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.353719 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.353806 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.355878 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.355880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.355880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.355888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.357621 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.358038 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.358039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.358040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.358095 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.358621 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.358623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.358623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.358766 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.358826 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.359108 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.359195 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:44.359602 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.351403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.189:33847 (local address 127.0.0.1:53846)
0808 07:15:44.351447 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.351456 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.351488 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.351641 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.351642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.351781 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.351837 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.351839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.352726 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.352727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.353410 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.353412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.353808 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.353809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.353884 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.354331 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.354345 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.355779 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.357711 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.357711 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.357712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.357927 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.358486 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.358486 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.358487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.358548 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.358852 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.358852 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.358889 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.358954 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.359027 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:15:44.360005 15669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:44.359910 15430 master_runner.cc:186] Error getting master registration for 127.22.2.189:33847: OK, instance_id { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" instance_seqno: 1786173344187438 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:15:44.362156 15430 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173344, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.188@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.190@KRBTEST.COM
I20260808 07:15:44.366912 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.360221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40534)
0808 07:15:44.360310 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.360327 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.360373 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.360487 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.360488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.360612 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.360633 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.360634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.361236 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.361237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.361700 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.361701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.362083 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.362083 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.362130 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.362520 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.362525 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.363719 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.365607 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.365608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.365608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.365838 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.366068 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.366068 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.366069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.366139 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.366486 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.366487 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.366533 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.366641 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.366779 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 07:15:44.367013 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.360276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.22.2.188:46299)
0808 07:15:44.360309 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.360311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.360357 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:15:44.360425 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.360426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.360434 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.360476 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.360712 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.361160 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.361779 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.362011 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.362086 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.363827 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.363828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.363828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.363834 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.365509 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.365923 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.365924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.365924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.365985 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.366216 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.366217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.366217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.366402 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.366476 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.366681 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.366850 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":15}
I20260808 07:15:44.369127 15669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907: Bootstrap starting.
I20260808 07:15:44.369791 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.359688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51990 (local address 127.22.2.190:42049)
0808 07:15:44.359728 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.359730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.359738 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:44.359812 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.359813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.359824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.359881 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.362734 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.363444 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.364217 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.364528 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.364593 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.367350 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.367351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.367352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.367358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.368654 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.368992 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.368993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.368994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.369049 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.369259 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.369259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.369260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.369375 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.369527 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.369605 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.369688 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:15:44.369904 15669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:44.370222 15669 log.cc:824] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:44.371106 15669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907: No bootstrap required, opened a new log
I20260808 07:15:44.373198 15669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } }
I20260808 07:15:44.373384 15669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1c4adf66d3423095bb4ba7bdca9907, State: Initialized, Role: FOLLOWER
I20260808 07:15:44.373770 15669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } }
I20260808 07:15:44.374171 15669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:44.374126 15692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.374348 15692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.2.189@KRBTEST.COM
W20260808 07:15:44.376379 15704 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:44.376432 15704 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:44.379634 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.370613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53850 (local address 127.22.2.189:33847)
0808 07:15:44.370653 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.370655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.370734 (+ 79us) server_negotiation.cc:407] Connection header received
0808 07:15:44.370820 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.370821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.370831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.370872 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.371411 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.371939 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.372575 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.373163 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.373247 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.375809 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.375810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.375810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.375830 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.378016 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.378427 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.378428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.378428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.378489 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.378680 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.378680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.378680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.378808 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.378947 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.379409 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.379491 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:44.377558 15596 master_runner.cc:186] Error getting master registration for 127.22.2.189:33847: OK, instance_id { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" instance_seqno: 1786173344187438 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:15:44.380159 15596 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:15:44.380777 22538 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:44.381588 15474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "de2ec9df307f4869872afb37e316568a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51990
I20260808 07:15:44.381733 15474 raft_consensus.cc:482] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:15:44.381786 15474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:44.382507 15474 raft_consensus.cc:504] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } }
I20260808 07:15:44.382761 15474 leader_election.cc:288] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [CANDIDATE]: Term 1 election: Requested vote from peers d155d4a0abea45a0bb4a17d8690ac238 (127.22.2.189:33847), 2e1c4adf66d3423095bb4ba7bdca9907 (127.22.2.188:46299)
I20260808 07:15:44.383219 15642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de2ec9df307f4869872afb37e316568a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2e1c4adf66d3423095bb4ba7bdca9907"
I20260808 07:15:44.383339 15642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:44.384266 15642 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2ec9df307f4869872afb37e316568a in term 1.
I20260808 07:15:44.384469 15447 leader_election.cc:302] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e1c4adf66d3423095bb4ba7bdca9907, de2ec9df307f4869872afb37e316568a; no voters:
I20260808 07:15:44.384601 15677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:44.384758 15677 raft_consensus.cc:686] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [term 1 LEADER]: Becoming Leader. State: Replica: de2ec9df307f4869872afb37e316568a, State: Running, Role: LEADER
I20260808 07:15:44.384872 15677 consensus_queue.cc:237] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } }
I20260808 07:15:44.385329 15706 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2ec9df307f4869872afb37e316568a. Latest consensus state: current_term: 1 leader_uuid: "de2ec9df307f4869872afb37e316568a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.385543 15706 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:44.386395 15709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:44.386582 15709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:44.388522 15642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [term 1 FOLLOWER]: Refusing update from remote peer de2ec9df307f4869872afb37e316568a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:44.388842 15677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:44.390261 15692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2ec9df307f4869872afb37e316568a. Latest consensus state: current_term: 1 leader_uuid: "de2ec9df307f4869872afb37e316568a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.390369 15692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:44.396903 15706 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de2ec9df307f4869872afb37e316568a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.397006 15706 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:44.397082 15714 mvcc.cc:205] Tried to move back new op lower bound from 7316166018612240384 to 7316166018601222144. Current Snapshot: MvccSnapshot[applied={T|T < 7316166018612240384}]
I20260808 07:15:44.397490 15692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de2ec9df307f4869872afb37e316568a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.397576 15692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:44.397945 15709 catalog_manager.cc:1347] Generated new cluster ID: b9d7accd3f034a4f97ac383cf4568aa5
I20260808 07:15:44.398002 15709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:44.403645 15718 mvcc.cc:205] Tried to move back new op lower bound from 7316166018612240384 to 7316166018601222144. Current Snapshot: MvccSnapshot[applied={T|T < 7316166018612240384 or (T in {7316166018612240384})}]
I20260808 07:15:44.407894 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.387835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.22.2.190:42049)
0808 07:15:44.387873 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.387875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.389397 (+ 1522us) server_negotiation.cc:407] Connection header received
0808 07:15:44.389586 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.389587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.389599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.389631 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.390990 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.391803 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.392846 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.393135 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.393196 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.404935 (+ 11739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.404937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.404937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.404946 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.406551 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.406962 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.406963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.406964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.407018 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.407230 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.407231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.407231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.407388 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.407464 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.407707 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.407799 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:44.408785 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.387491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.22.2.188:46299)
0808 07:15:44.387530 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.387532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.391062 (+ 3530us) server_negotiation.cc:407] Connection header received
0808 07:15:44.391148 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.391149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.391159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.391187 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.392873 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.393531 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.394537 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.394875 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.394920 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.404171 (+ 9251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.404173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.404173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.404182 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.406702 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.407400 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.407401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.407401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.407455 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.407845 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.407846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.407847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.407977 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.408058 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.408618 (+ 560us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.408702 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:44.416687 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.388223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.22.2.189:33847)
0808 07:15:44.388373 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.388375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.395306 (+ 6931us) server_negotiation.cc:407] Connection header received
0808 07:15:44.395362 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.395363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.395376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.395426 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.404720 (+ 9294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.405449 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.411128 (+ 5679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.411454 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.411491 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.413653 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.413654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.413654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.413662 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.415180 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.415740 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.415741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.415741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.415802 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.416060 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.416061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.416062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.416176 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.416240 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.416524 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.416597 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 07:15:44.421777 15588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } attempt: 3
Aug 08 07:15:44 dist-test-slave-6k22 krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173344, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.2.189@KRBTEST.COM for kudu/127.22.2.188@KRBTEST.COM
I20260808 07:15:44.430179 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.422048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.2.188:46299 (local address 127.0.0.1:40546)
0808 07:15:44.422127 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:44.422136 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:44.422166 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:44.422406 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:44.422407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:44.422547 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:44.422583 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.422584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.423927 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.423928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.424675 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:44.424677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.425209 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.425210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:44.425273 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.425827 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:44.425833 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:44.427262 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:44.429001 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.429002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.429002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.429204 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.429442 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:44.429443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:44.429443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:44.429513 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:44.429861 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:44.429862 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:44.429916 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 07:15:44.429983 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:15:44.430078 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 07:15:44.430825 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:44.422226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.22.2.188:46299)
0808 07:15:44.422277 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.422278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.422284 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:15:44.422316 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.422317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.422326 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.422369 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.422655 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.423364 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.424786 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:44.425089 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:44.425138 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:44.427362 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:44.427363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:44.427363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:44.427370 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:44.428899 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.429288 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.429289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.429290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.429358 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:44.429585 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:44.429586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:44.429586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:44.429745 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:44.429803 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:44.430647 (+ 844us) server_negotiation.cc:299] Negotiation successful
0808 07:15:44.430726 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:15:44.431409 15709 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:44.432076 15709 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:44.432533 15588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238: Bootstrap starting.
I20260808 07:15:44.433216 15588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:44.433473 15588 log.cc:824] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:44.435914 15588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238: No bootstrap required, opened a new log
I20260808 07:15:44.437721 15588 raft_consensus.cc:348] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } }
I20260808 07:15:44.437846 15588 raft_consensus.cc:729] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d155d4a0abea45a0bb4a17d8690ac238, State: Initialized, Role: FOLLOWER
I20260808 07:15:44.437911 15588 consensus_queue.cc:260] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } }
I20260808 07:15:44.438196 15588 sys_catalog.cc:564] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:44.439492 15555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de2ec9df307f4869872afb37e316568a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d155d4a0abea45a0bb4a17d8690ac238"
I20260808 07:15:44.439656 15554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:44.440445 15709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a: Generated new TSK 0
I20260808 07:15:44.440644 15709 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:44.441287 15554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [term 1 FOLLOWER]: Refusing update from remote peer de2ec9df307f4869872afb37e316568a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:44.441596 15706 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:44.442263 15555 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2ec9df307f4869872afb37e316568a in term 1.
W20260808 07:15:44.445618 15736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:44.445674 15736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:44.445760 15740 sys_catalog.cc:455] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2ec9df307f4869872afb37e316568a. Latest consensus state: current_term: 1 leader_uuid: "de2ec9df307f4869872afb37e316568a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.445832 15740 sys_catalog.cc:458] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:44.445878 15740 sys_catalog.cc:455] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de2ec9df307f4869872afb37e316568a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.445933 15740 sys_catalog.cc:458] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:44.446722 15723 sys_catalog.cc:455] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de2ec9df307f4869872afb37e316568a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.446801 15723 sys_catalog.cc:458] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:44.447178 15738 mvcc.cc:205] Tried to move back new op lower bound from 7316166018822475776 to 7316166018601222144. Current Snapshot: MvccSnapshot[applied={T|T < 7316166018612240384}]
W20260808 07:15:44.470328 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.468649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.22.2.189:33847)
0808 07:15:44.468695 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.468697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.469638 (+ 941us) server_negotiation.cc:407] Connection header received
0808 07:15:44.469746 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.469747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.469758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.469789 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.470256 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53864: BlockingRecv error: recv got EOF from 127.0.0.1:53864 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:15:44.471101 15504 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.469049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52016 (local address 127.22.2.190:42049)
0808 07:15:44.469089 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.469091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.470577 (+ 1486us) server_negotiation.cc:407] Connection header received
0808 07:15:44.470673 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.470674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.470685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.470714 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.471036 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52016: BlockingRecv error: recv got EOF from 127.0.0.1:52016 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:15:44.471841 15674 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.469367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.22.2.188:46299)
0808 07:15:44.469405 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.469406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.471320 (+ 1914us) server_negotiation.cc:407] Connection header received
0808 07:15:44.471419 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.471420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.471431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.471460 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.471771 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40552: BlockingRecv error: recv got EOF from 127.0.0.1:40552 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:15:44.474267 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.473036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.22.2.189:33847)
0808 07:15:44.473070 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.473072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.473657 (+ 585us) server_negotiation.cc:407] Connection header received
0808 07:15:44.473763 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.473764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.473782 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.473811 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.474221 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53878: BlockingRecv error: recv got EOF from 127.0.0.1:53878 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:44.477167 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.476425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53890 (local address 127.22.2.189:33847)
0808 07:15:44.476459 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.476460 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.476690 (+ 230us) server_negotiation.cc:407] Connection header received
0808 07:15:44.476781 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.476782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.476791 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.476833 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.477123 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53890: BlockingRecv error: recv got EOF from 127.0.0.1:53890 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:15:44.480892 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.480174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53896 (local address 127.22.2.189:33847)
0808 07:15:44.480206 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.480207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.480430 (+ 223us) server_negotiation.cc:407] Connection header received
0808 07:15:44.480520 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.480521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.480530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.480558 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.480852 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53896: BlockingRecv error: recv got EOF from 127.0.0.1:53896 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 07:15:44.486665 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.485888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.22.2.189:33847)
0808 07:15:44.485919 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.485920 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.486186 (+ 266us) server_negotiation.cc:407] Connection header received
0808 07:15:44.486292 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.486292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.486302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.486330 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.486622 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53908: BlockingRecv error: recv got EOF from 127.0.0.1:53908 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 07:15:44.496614 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.495767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.22.2.189:33847)
0808 07:15:44.495811 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.495813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.496073 (+ 260us) server_negotiation.cc:407] Connection header received
0808 07:15:44.496186 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.496187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.496197 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.496227 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.496557 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53912: BlockingRecv error: recv got EOF from 127.0.0.1:53912 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:15:44.514802 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.513867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53928 (local address 127.22.2.189:33847)
0808 07:15:44.513912 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.513914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.514191 (+ 277us) server_negotiation.cc:407] Connection header received
0808 07:15:44.514290 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.514291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.514302 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.514334 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.514739 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53928: BlockingRecv error: recv got EOF from 127.0.0.1:53928 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:15:44.549281 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.548250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.22.2.189:33847)
0808 07:15:44.548303 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.548306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.548630 (+ 324us) server_negotiation.cc:407] Connection header received
0808 07:15:44.548743 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.548744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.548758 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.548791 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.549220 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53932: BlockingRecv error: recv got EOF from 127.0.0.1:53932 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 07:15:44.615707 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.614765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53940 (local address 127.22.2.189:33847)
0808 07:15:44.614846 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.614849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.614990 (+ 141us) server_negotiation.cc:407] Connection header received
0808 07:15:44.615131 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.615132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.615164 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.615249 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.615639 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53940: BlockingRecv error: recv got EOF from 127.0.0.1:53940 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 07:15:44.745556 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:44.744850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53946 (local address 127.22.2.189:33847)
0808 07:15:44.744919 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:44.744924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:44.745031 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:15:44.745079 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:44.745079 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:44.745089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:44.745153 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:44.745476 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53946: BlockingRecv error: recv got EOF from 127.0.0.1:53946 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:15:44.829478 15706 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de2ec9df307f4869872afb37e316568a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2ec9df307f4869872afb37e316568a" member_type: VOTER last_known_addr { host: "127.22.2.190" port: 42049 } } peers { permanent_uuid: "d155d4a0abea45a0bb4a17d8690ac238" member_type: VOTER last_known_addr { host: "127.22.2.189" port: 33847 } } peers { permanent_uuid: "2e1c4adf66d3423095bb4ba7bdca9907" member_type: VOTER last_known_addr { host: "127.22.2.188" port: 46299 } } }
I20260808 07:15:44.829663 15706 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2ec9df307f4869872afb37e316568a [sys.catalog]: This master's current role is: LEADER
W20260808 07:15:45.003712 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:45.002861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53948 (local address 127.22.2.189:33847)
0808 07:15:45.002938 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:45.002941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:45.003055 (+ 114us) server_negotiation.cc:407] Connection header received
0808 07:15:45.003146 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:45.003152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:45.003163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:45.003226 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:45.003645 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53948: BlockingRecv error: recv got EOF from 127.0.0.1:53948 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:15:45.376897 15704 catalog_manager.cc:1259] Loaded cluster ID: b9d7accd3f034a4f97ac383cf4568aa5
I20260808 07:15:45.377012 15704 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907: loading cluster ID for follower catalog manager: success
I20260808 07:15:45.377867 15704 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907: acquiring CA information for follower catalog manager: success
I20260808 07:15:45.378189 15704 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e1c4adf66d3423095bb4ba7bdca9907: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:15:45.446171 15736 catalog_manager.cc:1259] Loaded cluster ID: b9d7accd3f034a4f97ac383cf4568aa5
I20260808 07:15:45.446305 15736 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238: loading cluster ID for follower catalog manager: success
I20260808 07:15:45.447278 15736 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238: acquiring CA information for follower catalog manager: success
I20260808 07:15:45.447703 15736 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d155d4a0abea45a0bb4a17d8690ac238: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:15:45.519743 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:45.516917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.22.2.189:33847)
0808 07:15:45.517078 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:45.517082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:45.517096 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:15:45.517132 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:45.517133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:45.517206 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:45.517324 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:45.517479 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.518050 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.518818 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.519022 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.519087 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:45.519255 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:45.519373 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:45.519445 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:45.519569 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 07:15:45.519626 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260808 07:15:45.523913 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:45.521526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40564 (local address 127.22.2.188:46299)
0808 07:15:45.521619 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:45.521621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:45.521628 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:45.521668 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:45.521668 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:45.521706 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:45.521779 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:45.521880 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.522367 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.523124 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.523335 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.523388 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:45.523506 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:45.523667 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:45.523726 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:45.523773 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 07:15:45.523792 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":49,"threads_started":1}
I20260808 07:15:45.528671 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:45.525922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53964 (local address 127.22.2.189:33847)
0808 07:15:45.526007 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:45.526009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:45.526017 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:45.526046 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:45.526047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:45.526058 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:45.526117 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:45.526246 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.527006 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.527914 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.528168 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.528231 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:45.528384 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:45.528420 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:45.528469 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:45.528558 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 07:15:45.528583 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 07:15:45.528888 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:45.525961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.22.2.188:46299)
0808 07:15:45.526003 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:45.526005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:45.526268 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:15:45.526338 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:45.526339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:45.526351 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:45.526401 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:45.526555 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.527005 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.528120 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.528396 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.528466 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:45.528611 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:45.528643 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:45.528672 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:45.528789 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:15:45.528816 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:45.529366 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:45.525749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.22.2.190:42049)
0808 07:15:45.525861 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:45.525864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:45.525982 (+ 118us) server_negotiation.cc:407] Connection header received
0808 07:15:45.526030 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:45.526031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:45.526087 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:45.526235 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:45.526411 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.527163 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.528167 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:45.528511 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:45.528572 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:45.528805 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:45.528932 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:45.529137 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:45.529234 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 07:15:45.529256 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260808 07:15:45.531231 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 15430
I20260808 07:15:45.542037 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 15508
I20260808 07:15:45.548280 22538 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyv8zyi/build/release/bin/kudu with pid 15596
2026-08-08T07:15:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2957 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5860 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (276815 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST